Developing event-driven microservices with event sourcing and CQRS (svcc, sv...Chris Richardson
Modern, cloud-native applications typically use a microservices architecture in conjunction with NoSQL and/or sharded relational databases. However, in order to successfully use this approach you need to solve some distributed data management problems including how to maintain consistency between multiple databases without using 2PC.
In this talk you will learn more about these issues and how to solve them by using an event-driven architecture. We will describe how event sourcing and Command Query Responsibility Segregation (CQRS) are a great way to realize an event-driven architecture. You will learn about a simple yet powerful approach for building, modern, scalable applications.
Clinical Data Repository vs. A Data Warehouse - Which Do You Need?Health Catalyst
It can be confusing to know whether or not your health system needs to add a data warehouse unless you understand how it’s different from a clinical data repository. A clinical data repository consolidates data from various clinical sources, such as an EMR, to provide a clinical view of patients. A data warehouse, in comparison, provides a single source of truth for all types of data pulled in from the many source systems across the enterprise. The data warehouse also has these benefits: a faster time to value, flexible architecture to make easy adjustments, reduction in waste and inefficiencies, reduced errors, standardized reports, decreased wait times for reports, data governance and security.
Database vs Data Warehouse: A Comparative ReviewHealth Catalyst
What are the differences between a database and a data warehouse? A database is any collection of data organized for storage, accessibility, and retrieval. A data warehouse is a type of database the integrates copies of transaction data from disparate source systems and provisions them for analytical use. The important distinction is that data warehouses are designed to handle analytics required for improving quality and costs in the new healthcare environment. A transactional database, like an EHR, doesn’t lend itself to analytics.
Building Microservice Systems Without Cooking Your Laptop: Going “Remocal” wi...Ambassador Labs
When you adopt microservices, containers, and cloud native development, the technologies and architectures may change, but the need for fast feedback doesn’t. Kubernetes enables us to deploy and run applications at scale, but whether you’re coding or testing applications, you want to be able to get work done quickly without spinning up all of your microservices locally and driving your laptop fans into high speed!
Join me for a tour of coding, testing, and shipping microservices using remote-to-local “remocal” tools and techniques. You will:
Understand the challenges with scaling container-based application development – i.e. you can only run so many microservices locally before minikube melts your laptop.
Learn when to use various types of development practices and tooling based on your use case and requirements for production realism, speed, and practicality.
Explore how to utilize containerized dependencies and Docker for testing, including for both apps and services you own and those you don’t.
Learn how Telepresence can enable “remocal” development, expanding your local machine and Docker Desktop out into a remote Kubernetes cluster.
Ship Week 1: Intro to Continuous Delivery and GitOps
When building cloud native applications, software developers are no longer just responsible for coding new features. In the next module of Summer of Kubernetes, our expert guides (with the help of some special guests) will cover how to safely and effectively ship software without disrupting end users. To do this you will:
✅ Understand the basics of continuous delivery and GitOps
✅ Learn about how K8s enables declarative CD (via the use of reconciliation loops)
At GOTO Amsterdam in 2019 I presented how to create an effective cloud native developer workflow. Two years later and many new developer technologies have come and gone, but I still hear daily from cloud developers about the pain and friction associated with building, debugging, and deploying to the cloud. In this talk I'll share my latest learning on how to bring the fun and productivity back into delivering Kubernetes-based software.
In this talk, you will:
- Learn why the core tenets of continuous delivery -- speed and safety -- must be considered in all parts of the cloud native SDLC
- Explore how cloud native coding benefits from thinking separately about the inner development loop, continuous integration, continuous deployment, observability, and analysis
- Understand how cloud native best practices and tooling fit together. Learn about artifact syncing (e.g. Skaffold), dev environment bridging (e.g. Telepresence), GitOps (e.g. Argo), and observability-focused monitoring (e.g. Prometheus, Jaeger)
- Explore the importance of cultivating an effective cloud platform and associated team of experts
- Walk away with an overview of tools that can help you develop and debug effectively when using Kubernetes
Webinar: Accelerate Your Inner Dev Loop for Kubernetes Services Ambassador Labs
Many turn to static duplicate dev environments to shorten the dev loop and isolate code tests, but those bring about additional issues. The idea of safely sharing a dev environment and seeing your code changes in action immediately before sharing them probably seems impossible.
Service Preview, powered by Telepresence and the Ambassador Edge Stack, is here to help! This capability enables you to preview changes immediately and test locally with your tool of choice, while sharing a development cluster.
In this 45-minute webinar, Abhay Saxena will demonstrate using Service Preview to have a fast inner development loop while fixing a bug in a microservice, including stepping through the code in a debugger while other developers continue working unaffected.
[Confoo Montreal 2020] From Grief to Growth: The 7 Stages of Observability - ...Ambassador Labs
In this case-study talk, we will share Brent’s journey through the adoption of modern observability practices as he operated an architecture of distributed services. Facing difficulties using application logs as the primary tool to debug performance and reliability issues? Learn how to improve your company toolkit and engineering habits using existing monitoring tools with the addition of distributed tracing.
https://confoo.ca/en/yul2020/session/from-grief-to-growth-the-7-stages-of-observability
Developing event-driven microservices with event sourcing and CQRS (svcc, sv...Chris Richardson
Modern, cloud-native applications typically use a microservices architecture in conjunction with NoSQL and/or sharded relational databases. However, in order to successfully use this approach you need to solve some distributed data management problems including how to maintain consistency between multiple databases without using 2PC.
In this talk you will learn more about these issues and how to solve them by using an event-driven architecture. We will describe how event sourcing and Command Query Responsibility Segregation (CQRS) are a great way to realize an event-driven architecture. You will learn about a simple yet powerful approach for building, modern, scalable applications.
Clinical Data Repository vs. A Data Warehouse - Which Do You Need?Health Catalyst
It can be confusing to know whether or not your health system needs to add a data warehouse unless you understand how it’s different from a clinical data repository. A clinical data repository consolidates data from various clinical sources, such as an EMR, to provide a clinical view of patients. A data warehouse, in comparison, provides a single source of truth for all types of data pulled in from the many source systems across the enterprise. The data warehouse also has these benefits: a faster time to value, flexible architecture to make easy adjustments, reduction in waste and inefficiencies, reduced errors, standardized reports, decreased wait times for reports, data governance and security.
Database vs Data Warehouse: A Comparative ReviewHealth Catalyst
What are the differences between a database and a data warehouse? A database is any collection of data organized for storage, accessibility, and retrieval. A data warehouse is a type of database the integrates copies of transaction data from disparate source systems and provisions them for analytical use. The important distinction is that data warehouses are designed to handle analytics required for improving quality and costs in the new healthcare environment. A transactional database, like an EHR, doesn’t lend itself to analytics.
Building Microservice Systems Without Cooking Your Laptop: Going “Remocal” wi...Ambassador Labs
When you adopt microservices, containers, and cloud native development, the technologies and architectures may change, but the need for fast feedback doesn’t. Kubernetes enables us to deploy and run applications at scale, but whether you’re coding or testing applications, you want to be able to get work done quickly without spinning up all of your microservices locally and driving your laptop fans into high speed!
Join me for a tour of coding, testing, and shipping microservices using remote-to-local “remocal” tools and techniques. You will:
Understand the challenges with scaling container-based application development – i.e. you can only run so many microservices locally before minikube melts your laptop.
Learn when to use various types of development practices and tooling based on your use case and requirements for production realism, speed, and practicality.
Explore how to utilize containerized dependencies and Docker for testing, including for both apps and services you own and those you don’t.
Learn how Telepresence can enable “remocal” development, expanding your local machine and Docker Desktop out into a remote Kubernetes cluster.
Ship Week 1: Intro to Continuous Delivery and GitOps
When building cloud native applications, software developers are no longer just responsible for coding new features. In the next module of Summer of Kubernetes, our expert guides (with the help of some special guests) will cover how to safely and effectively ship software without disrupting end users. To do this you will:
✅ Understand the basics of continuous delivery and GitOps
✅ Learn about how K8s enables declarative CD (via the use of reconciliation loops)
At GOTO Amsterdam in 2019 I presented how to create an effective cloud native developer workflow. Two years later and many new developer technologies have come and gone, but I still hear daily from cloud developers about the pain and friction associated with building, debugging, and deploying to the cloud. In this talk I'll share my latest learning on how to bring the fun and productivity back into delivering Kubernetes-based software.
In this talk, you will:
- Learn why the core tenets of continuous delivery -- speed and safety -- must be considered in all parts of the cloud native SDLC
- Explore how cloud native coding benefits from thinking separately about the inner development loop, continuous integration, continuous deployment, observability, and analysis
- Understand how cloud native best practices and tooling fit together. Learn about artifact syncing (e.g. Skaffold), dev environment bridging (e.g. Telepresence), GitOps (e.g. Argo), and observability-focused monitoring (e.g. Prometheus, Jaeger)
- Explore the importance of cultivating an effective cloud platform and associated team of experts
- Walk away with an overview of tools that can help you develop and debug effectively when using Kubernetes
Webinar: Accelerate Your Inner Dev Loop for Kubernetes Services Ambassador Labs
Many turn to static duplicate dev environments to shorten the dev loop and isolate code tests, but those bring about additional issues. The idea of safely sharing a dev environment and seeing your code changes in action immediately before sharing them probably seems impossible.
Service Preview, powered by Telepresence and the Ambassador Edge Stack, is here to help! This capability enables you to preview changes immediately and test locally with your tool of choice, while sharing a development cluster.
In this 45-minute webinar, Abhay Saxena will demonstrate using Service Preview to have a fast inner development loop while fixing a bug in a microservice, including stepping through the code in a debugger while other developers continue working unaffected.
[Confoo Montreal 2020] From Grief to Growth: The 7 Stages of Observability - ...Ambassador Labs
In this case-study talk, we will share Brent’s journey through the adoption of modern observability practices as he operated an architecture of distributed services. Facing difficulties using application logs as the primary tool to debug performance and reliability issues? Learn how to improve your company toolkit and engineering habits using existing monitoring tools with the addition of distributed tracing.
https://confoo.ca/en/yul2020/session/from-grief-to-growth-the-7-stages-of-observability
[Confoo Montreal 2020] Build Your Own Serverless with Knative - Alex GervaisAmbassador Labs
Google Cloud Run’s use of Knative introduced a portable Serverless solution built on top of Kubernetes. In this talk, we’ll recap the basic guidelines, use cases, and benefits of a Serverless architecture. Getting up and started, you will learn to take advantage of containers and the Ambassador API Gateway to serve event-driven application workloads and save costs using your existing Kubernetes resources.
https://confoo.ca/en/yul2020/session/build-your-own-serverless-with-knative
[QCon London 2020] The Future of Cloud Native API Gateways - Richard LiAmbassador Labs
The introduction of microservices, Kubernetes, and cloud technology has provided many benefits for developers. However, the age-old problem of getting user traffic routed correctly to the API of your backend applications can still be an issue, and may be complicated with the adoption of cloud native approaches: applications are now composed of multiple (micro)services that are built and released by independent teams; the underlying infrastructure is dynamically changing; services support multiple protocols, from HTTP/JSON to WebSockets and gRPC, and more; and many API endpoints require custom configuration of cross-cutting concerns, such as authn/z, rate limiting, and retry policies.
A cloud native API gateway is on the critical path of all requests, and also on the critical path for the workflow of any developer that is releasing functionality. Join this session to learn about the underlying technology and the required changes in engineering workflows. Key takeaways will include:
A brief overview of the evolution of API gateways over the past ten years, and how the original problems being solved have shifted in relation to cloud native technologies and workflow
Two important challenges when using an API gateway within Kubernetes: scaling the developer workflow; and supporting multiple architecture styles and protocols
Strategies for exposing Kubernetes services and APIs at the edge of your system
Insight into the (potential) future of cloud native API gateways
https://qconlondon.com/london2020/presentation/future-cloud-native-api-gateways
What's New in the Ambassador Edge Stack 1.0? Ambassador Labs
Before Kubernetes, the boundary between your users and your monolithic application was simple to manage. Now with Kubernetes, managing the edge has become dynamic and complex. More developers are involved, there are exponentially more edge operations, and each microservice has diverse requirements.
To fully capitalize on the benefits of Kubernetes, you need to provide a solution that supports the autonomy of application developers, the various requirements of your microservices, and your ability to scale.
You no longer need an API Gateway - you need a self-service, comprehensive edge stack.
In this 40 minute webinar on January 30th, we will discuss and demo the new functionality available with the Ambassador Edge Stack.
Edge Policy Console- graphical UI to visualize and manage all of your edge policies
Security Features- automatic TLS setup via ACME integration, OAuth/OpenID Connect integration, rate limiting, and fine-grained access control
Developer Onboarding- API catalog, Swagger/OpenAPI documentation support, and a fully customizable developer portal
Webinar: Effective Management of APIs and the Edge when Adopting Kubernetes Ambassador Labs
As you adopt Kubernetes, the requirements for your edge change. You now have teams working on multiple services all with different requirements. How can you make sure your edge is Kubernetes-ready?
[KubeCon NA 2018] Telepresence Deep Dive Session - Rafael Schloming & Luke Sh...Ambassador Labs
One of the challenges facing Telepresence is growing the contributor community. It’s a complex application that requires a good understanding of OS networking, VPNs, Kubernetes, and everything in between. We’ll kick off this meeting with a general architectural overview of Telepresence. We’ll talk about how we’ve managed the project to date, and our investments to make it easier. We want to then turn it over for an interactive discussion with participants to see what we can do to make it easier to contribute and grow the Telepresence community.
[KubeCon NA 2018] Effective Kubernetes Develop: Turbocharge Your Dev Loop - P...Ambassador Labs
Every software development cycle is rife with inefficiency. Seasoned devs know the pain of getting access to essential remote systems, waiting for tests to run (and then fail), or debugging with only log files. This talk teaches you how to best leverage Kubernetes, remote infrastructure and related tooling to create a dev cycle that maximizes velocity and minimizes developer friction and frustration.
Using tools such as Kubernetes, Docker and Telepresence, I will walk attendees through several advanced techniques that can be used to produce an effective developer experience and optimized dev loop. The goal of this is to eliminate many sources of frustrating inefficiency and reduce cycle time between releases. I will demonstrate how to incrementally adopt some of these techniques and how to approach introducing new and unfamiliar technology and techniques to skeptical dev teams.
The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and ...Ambassador Labs
Modern cloud applications today are built as distributed microservices. These microservices talk to each other over L7 protocols: HTTP, gRPC, Redis, Kafka, and more. In this world, L7 proxies have assumed a crucial role in managing and observing L7 protocols. In this talk, I’ll discuss the evolution of service architectures, the role L7 proxies play in this world, and how there is now a battle raging between Envoy Proxy, HAProxy, and NGINX. I’ll wrap by talking about why we chose Envoy Proxy as the anchor of our Ambassador API Gateway and show how that has enabled a number of new capabilities.
The Simply Complex Task of Implementing Kubernetes Ingress - Velocity NYCAmbassador Labs
Getting traffic into a Kubernetes cluster should be simple, but it’s not. Richard Li explains how software architectures have evolved to take advantage of Kubernetes and discusses the implications that these changes have on ingress. Richard then covers some of the nuances of modern ingress, including authentication, resilience, and observability at the edge, explores how Kubernetes handles ingress today, with NodePorts, LoadBalancers, and ingress controllers, and shares his experience and lessons learned from using several real-world implementations of ingress on Kubernetes.
KubeCon NA 2017: Ambassador and Envoy (Envoy Salon)Ambassador Labs
Ambassador is an open source Kubernetes-native API Gateway built on the Envoy proxy. We talked about why and how we built Ambassador during the Envoy salon at KubeCon.
What’s the key to successfully adopting microservices on Kubernetes?
Building a development workflow that helps developers code faster.
In this webinar, we introduce the principles of a cloud-native development workflow where individual teams build and ship software independently from each other.
QCon SF 2017 - Microservices: Service-Oriented DevelopmentAmbassador Labs
Conventional wisdom is that microservices is an architecture that is the spiritual successor to service-oriented architecture. While true, this myopic view of microservices ignores some of the profound workflow shifts in today’s microservices organizations.
The reality is that microservices is an architecture _and_ workflow. In this talk, we’ll introduce the workflow of service-oriented development. Rafael will talk about how the real goal of microservices is to break up a monolithic development workflow. We’ll show you how, by breaking up your workflow, you can build software that lets you move fast and make things.
DevOps Days Boston 2017: Real-world Kubernetes for DevOpsAmbassador Labs
DevOps Days Boston 2017
Microservices is an increasingly popular approach to building cloud-native applications. Dozens of new technologies that streamline adopting microservices development such as Docker, Kubernetes, and Envoy have been released over the past few years. But how do you actually use these technologies together to develop, deploy, and run microservices?
In this presentation, we’ll cover the nuances of deploying containerized applications on Kubernetes, including creating a Kubernetes manifest, debugging and logging, and how to build an automated continuous deployment pipeline. Then, we’ll do a brief tour of some of the advanced concepts related to microservices, including service mesh, canary deployments, resilience, and security.
DevOps Days Boston 2017: Developer first workflows for KubernetesAmbassador Labs
Kubernetes is a powerful, operational platform for containerized applications. However, the developer workflow on Kubernetes – how you code, deploy, update, and monitor your services – is much less mature.
How should you lay out your Git repo? How do you create loosely coupled services? How do you support deploying your service at any time?
In this talk, we’ll talk about these questions and more. We’ll discuss the journey towards a rapid development workflow, discuss best practices, and, talk about the process we followed to get to a rapid development workflow.
DevOps Days Boston 2017
[Confoo Montreal 2020] Build Your Own Serverless with Knative - Alex GervaisAmbassador Labs
Google Cloud Run’s use of Knative introduced a portable Serverless solution built on top of Kubernetes. In this talk, we’ll recap the basic guidelines, use cases, and benefits of a Serverless architecture. Getting up and started, you will learn to take advantage of containers and the Ambassador API Gateway to serve event-driven application workloads and save costs using your existing Kubernetes resources.
https://confoo.ca/en/yul2020/session/build-your-own-serverless-with-knative
[QCon London 2020] The Future of Cloud Native API Gateways - Richard LiAmbassador Labs
The introduction of microservices, Kubernetes, and cloud technology has provided many benefits for developers. However, the age-old problem of getting user traffic routed correctly to the API of your backend applications can still be an issue, and may be complicated with the adoption of cloud native approaches: applications are now composed of multiple (micro)services that are built and released by independent teams; the underlying infrastructure is dynamically changing; services support multiple protocols, from HTTP/JSON to WebSockets and gRPC, and more; and many API endpoints require custom configuration of cross-cutting concerns, such as authn/z, rate limiting, and retry policies.
A cloud native API gateway is on the critical path of all requests, and also on the critical path for the workflow of any developer that is releasing functionality. Join this session to learn about the underlying technology and the required changes in engineering workflows. Key takeaways will include:
A brief overview of the evolution of API gateways over the past ten years, and how the original problems being solved have shifted in relation to cloud native technologies and workflow
Two important challenges when using an API gateway within Kubernetes: scaling the developer workflow; and supporting multiple architecture styles and protocols
Strategies for exposing Kubernetes services and APIs at the edge of your system
Insight into the (potential) future of cloud native API gateways
https://qconlondon.com/london2020/presentation/future-cloud-native-api-gateways
What's New in the Ambassador Edge Stack 1.0? Ambassador Labs
Before Kubernetes, the boundary between your users and your monolithic application was simple to manage. Now with Kubernetes, managing the edge has become dynamic and complex. More developers are involved, there are exponentially more edge operations, and each microservice has diverse requirements.
To fully capitalize on the benefits of Kubernetes, you need to provide a solution that supports the autonomy of application developers, the various requirements of your microservices, and your ability to scale.
You no longer need an API Gateway - you need a self-service, comprehensive edge stack.
In this 40 minute webinar on January 30th, we will discuss and demo the new functionality available with the Ambassador Edge Stack.
Edge Policy Console- graphical UI to visualize and manage all of your edge policies
Security Features- automatic TLS setup via ACME integration, OAuth/OpenID Connect integration, rate limiting, and fine-grained access control
Developer Onboarding- API catalog, Swagger/OpenAPI documentation support, and a fully customizable developer portal
Webinar: Effective Management of APIs and the Edge when Adopting Kubernetes Ambassador Labs
As you adopt Kubernetes, the requirements for your edge change. You now have teams working on multiple services all with different requirements. How can you make sure your edge is Kubernetes-ready?
[KubeCon NA 2018] Telepresence Deep Dive Session - Rafael Schloming & Luke Sh...Ambassador Labs
One of the challenges facing Telepresence is growing the contributor community. It’s a complex application that requires a good understanding of OS networking, VPNs, Kubernetes, and everything in between. We’ll kick off this meeting with a general architectural overview of Telepresence. We’ll talk about how we’ve managed the project to date, and our investments to make it easier. We want to then turn it over for an interactive discussion with participants to see what we can do to make it easier to contribute and grow the Telepresence community.
[KubeCon NA 2018] Effective Kubernetes Develop: Turbocharge Your Dev Loop - P...Ambassador Labs
Every software development cycle is rife with inefficiency. Seasoned devs know the pain of getting access to essential remote systems, waiting for tests to run (and then fail), or debugging with only log files. This talk teaches you how to best leverage Kubernetes, remote infrastructure and related tooling to create a dev cycle that maximizes velocity and minimizes developer friction and frustration.
Using tools such as Kubernetes, Docker and Telepresence, I will walk attendees through several advanced techniques that can be used to produce an effective developer experience and optimized dev loop. The goal of this is to eliminate many sources of frustrating inefficiency and reduce cycle time between releases. I will demonstrate how to incrementally adopt some of these techniques and how to approach introducing new and unfamiliar technology and techniques to skeptical dev teams.
The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and ...Ambassador Labs
Modern cloud applications today are built as distributed microservices. These microservices talk to each other over L7 protocols: HTTP, gRPC, Redis, Kafka, and more. In this world, L7 proxies have assumed a crucial role in managing and observing L7 protocols. In this talk, I’ll discuss the evolution of service architectures, the role L7 proxies play in this world, and how there is now a battle raging between Envoy Proxy, HAProxy, and NGINX. I’ll wrap by talking about why we chose Envoy Proxy as the anchor of our Ambassador API Gateway and show how that has enabled a number of new capabilities.
The Simply Complex Task of Implementing Kubernetes Ingress - Velocity NYCAmbassador Labs
Getting traffic into a Kubernetes cluster should be simple, but it’s not. Richard Li explains how software architectures have evolved to take advantage of Kubernetes and discusses the implications that these changes have on ingress. Richard then covers some of the nuances of modern ingress, including authentication, resilience, and observability at the edge, explores how Kubernetes handles ingress today, with NodePorts, LoadBalancers, and ingress controllers, and shares his experience and lessons learned from using several real-world implementations of ingress on Kubernetes.
KubeCon NA 2017: Ambassador and Envoy (Envoy Salon)Ambassador Labs
Ambassador is an open source Kubernetes-native API Gateway built on the Envoy proxy. We talked about why and how we built Ambassador during the Envoy salon at KubeCon.
What’s the key to successfully adopting microservices on Kubernetes?
Building a development workflow that helps developers code faster.
In this webinar, we introduce the principles of a cloud-native development workflow where individual teams build and ship software independently from each other.
QCon SF 2017 - Microservices: Service-Oriented DevelopmentAmbassador Labs
Conventional wisdom is that microservices is an architecture that is the spiritual successor to service-oriented architecture. While true, this myopic view of microservices ignores some of the profound workflow shifts in today’s microservices organizations.
The reality is that microservices is an architecture _and_ workflow. In this talk, we’ll introduce the workflow of service-oriented development. Rafael will talk about how the real goal of microservices is to break up a monolithic development workflow. We’ll show you how, by breaking up your workflow, you can build software that lets you move fast and make things.
DevOps Days Boston 2017: Real-world Kubernetes for DevOpsAmbassador Labs
DevOps Days Boston 2017
Microservices is an increasingly popular approach to building cloud-native applications. Dozens of new technologies that streamline adopting microservices development such as Docker, Kubernetes, and Envoy have been released over the past few years. But how do you actually use these technologies together to develop, deploy, and run microservices?
In this presentation, we’ll cover the nuances of deploying containerized applications on Kubernetes, including creating a Kubernetes manifest, debugging and logging, and how to build an automated continuous deployment pipeline. Then, we’ll do a brief tour of some of the advanced concepts related to microservices, including service mesh, canary deployments, resilience, and security.
DevOps Days Boston 2017: Developer first workflows for KubernetesAmbassador Labs
Kubernetes is a powerful, operational platform for containerized applications. However, the developer workflow on Kubernetes – how you code, deploy, update, and monitor your services – is much less mature.
How should you lay out your Git repo? How do you create loosely coupled services? How do you support deploying your service at any time?
In this talk, we’ll talk about these questions and more. We’ll discuss the journey towards a rapid development workflow, discuss best practices, and, talk about the process we followed to get to a rapid development workflow.
DevOps Days Boston 2017
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
The infamous Mallox is the digital Robin Hoods of our time, except they steal from everyone and give to themselves. Since mid-2021, they've been playing hide and seek with unsecured Microsoft SQL servers, encrypting data, and then graciously offering to give it back for a modest Bitcoin donation.
Mallox decided to go shopping for new malware toys, adding the Remcos RAT, BatCloak, and a sprinkle of Metasploit to their collection. They're now playing a game of "Catch me if you can" with antivirus software, using their FUD obfuscator packers to turn their ransomware into the digital equivalent of a ninja.
-------
This document provides a analysis of the Target Company ransomware group, also known as Smallpox, which has been rapidly evolving since its first identification in June 2021.
The analysis delves into various aspects of the group's operations, including its distinctive practice of appending targeted organizations' names to encrypted files, the evolution of its encryption algorithms, and its tactics for establishing persistence and evading defenses.
The insights gained from this analysis are crucial for informing defense strategies and enhancing preparedness against such evolving cyber threats.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlPeter Udo Diehl
I'm excited to share my latest predictions on how AI, robotics, and other technological advancements will reshape industries in the coming years. The slides explore the exponential growth of computational power, the future of AI and robotics, and their profound impact on various sectors.
Why this matters:
The success of new products and investments hinges on precise timing and foresight into emerging categories. This deck equips founders, VCs, and industry leaders with insights to align future products with upcoming tech developments. These insights enhance the ability to forecast industry trends, improve market timing, and predict competitor actions.
Highlights:
▪ Exponential Growth in Compute: How $1000 will soon buy the computational power of a human brain
▪ Scaling of AI Models: The journey towards beyond human-scale models and intelligent edge computing
▪ Transformative Technologies: From advanced robotics and brain interfaces to automated healthcare and beyond
▪ Future of Work: How automation will redefine jobs and economic structures by 2040
With so many predictions presented here, some will inevitably be wrong or mistimed, especially with potential external disruptions. For instance, a conflict in Taiwan could severely impact global semiconductor production, affecting compute costs and related advancements. Nonetheless, these slides are intended to guide intuition on future technological trends.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Lyft's Envoy: From Monolith to Service Mesh - Matt Klein, Lyft
1. Lyft's Envoy: From monolith to service mesh
Matt Klein, Software Engineer @Lyft
2. Agenda
● Historical Lyft SoA architecture
● State of SoA networking in industry
● What is Envoy
● Envoy deployment @Lyft
● Envoy future directions
● Q&A
3. Lyft ~3.5 years ago
PHP / Apache
monolith
MongoDB
InternetClients AWS ELB
Simple! No SoA!
4. Lyft ~2 years ago
PHP / Apache
monolith
(+haproxy/nsq)
MongoDB
Internet
Clients
AWS external
ELB
DynamoDB
AWS internal
ELBs
Python services
Not simple! SoA! With monolith!
(and some haproxy/nsq)
5. Agenda
● Historical Lyft SoA architecture
● State of SoA networking in industry
● What is Envoy
● Envoy deployment @Lyft
● Envoy future directions
● Q&A
6. State of SoA networking in industry
● Languages and frameworks.
● Per language libraries for service calls.
● Protocols (HTTP/1, HTTP/2, gRPC, databases, caching, etc.).
● Infrastructures (IaaS, CaaS, on premise, etc.).
● Intermediate load balancers (AWS ELB, F5, etc.).
● Observability output (stats, tracing, and logging).
● Implementations (often partial) of retry, circuit breaking, rate limiting,
timeouts, and other distributed systems best practices.
● Authentication and Authorization.
7. State of SoA networking in industry
A really big and confusing mess...
8. State of SoA networking in industry
● Likely already in a world of hurt or rapidly approaching that point.
● Debugging is difficult or impossible (each application exposes different stats
and logs with no tracing).
● Limited visibility into infra components such as hosted load balancers,
databases, caches, network topologies, etc.).
● Multiple and partial implementations of circuit breaking, retry, and rate limiting
(If I had a $ for every time someone told me that retries are “easy” …).
● Furthermore, if you do have a good solution, you are likely using a library and
are locked into a particular technology stack essentially forever.
● Libraries are incredibly painful to upgrade. (Think CVEs).
9. State of SoA networking in industry
● Ultimately, robust observability and easy debugging are everything.
● As SoAs become more complicated, it is critical that we provide a common
solution to all of these problems or developer productivity grinds to a halt (and
the site goes down … often).
Can we do better?
10. Agenda
● Historical Lyft SoA architecture
● State of SoA networking in industry
● What is Envoy
● Envoy deployment @Lyft
● Envoy future directions
● Q&A
11. What is Envoy
The network should be transparent to applications. When
network and application problems do occur it should be easy to
determine the source of the problem.
This sounds great! But it turns out it’s really, really hard.
12. What is Envoy
● Out of process architecture: Let’s do a lot of really hard stuff in one place and
allow application developers to focus on business logic.
● Modern C++11 code base: Fast and productive.
● L3/L4 filter architecture: A TCP proxy at its core. Can be used for things other
than HTTP (e.g., MongoDB, redis, stunnel replacement, TCP rate limiter, etc.).
● HTTP L7 filter architecture: Make it easy to plug in different functionality.
● HTTP/2 first! (Including gRPC and a nifty gRPC HTTP/1.1 bridge).
● Service discovery and active health checking.
● Advanced load balancing: Retry, timeouts, circuit breaking, rate limiting,
shadowing, etc.
● Best in class observability: stats, logging, and tracing.
● Edge proxy: routing and TLS.
13. Envoy service to service topology
Service Cluster
Envoy
Service
Discovery
Service Cluster
Envoy
Service
External Services
HTTP/2
REST / GRPC
14. Envoy edge proxy topology
“Front” Envoy
Edge Proxy
Region #1
InternetExternal Clients
HTTP/1.1, HTTP/2, TLS
“Front” Envoy
Edge Proxy
Region #2
Private
Infra
HTTP/2, TLS, Client Auth
16. Eventually consistent service discovery
● Fully consistent service discovery systems are very popular (ZK, etcd, consul,
etc.).
● In practice they are hard to run at scale.
● Service discovery is actually an eventually consistent problem. Let’s
recognize that and design for it.
● Envoy is designed from the get go to treat service discovery as lossy.
● Active health checking used in combination with service discovery to produce a
routable overlay.
Discovery Status HC OK HC Failed
Discovered Route Don’t Route
Absent Route Don’t Route / Delete
17. Advanced load balancing
● Different service discovery types.
● Zone aware least request load balancing.
● Dynamic stats: Per zone, canary specific stats, etc.
● Circuit breaking: Max connections, requests, and retries.
● Rate limiting: Integration with global rate limit service.
● Shadowing: Fork traffic to a test cluster.
● Retries: HTTP router has built in retry capability with different policies.
● Timeouts: Both “outer” (including all retries) and “inner” (per try) timeouts.
● Outlier detection: Consecutive 5xx
● Deploy control: Blue/green, canary, etc.
18. Observability
● Observability is by far the most important thing that Envoy provides.
● Having all SoA traffic transit through Envoy gives us a single place where we
can:
○ Produce consistent statistics for every hop
○ Create and propagate a stable request ID
○ Consistent logging
○ Distributed tracing
23. Performance matters for a service proxy
● Throughput is important ultimately for cost and operational scaling, but for many
companies developer time is worth more than infra costs.
● BUT: Latency and predictability is what matters. And in particular tail latency
(P99+).
● We already deal with incredibly confusing deployments (virtual IaaS, multiple
languages and runtimes, languages that use GC, etc.). All of these niceties
improve productivity and reduce upfront dev costs, but they make debugging
really difficult.
● What is leading to sporadic error? The IaaS? The app? GC?
● Ability to reason about overall performance and reliability is critical.
24. Performance matters for a service proxy
● A service proxy provides invaluable benefits, but if the proxy itself has tail
latencies that are hard to reason about, most of the debugging benefits go out
the window and you are back to square one.
● Anyone that is trying to sell you a service infra that does not consider the above
points is selling you a dream...
25. Agenda
● Historical Lyft SoA architecture
● State of SoA networking in industry
● What is Envoy
● Envoy deployment @Lyft
● Envoy future directions
● Q&A
26. Envoy deployment @Lyft
● > 100 services.
● > 10,000 hosts.
● > 2,000,000 RPS.
● All service to service traffic (REST and gRPC).
● Use gRPC bridge to unlock Python and PHP clients.
● MongoDB proxy.
● DynamoDB proxy.
● External service proxy (AWS and other partners).
● Kibana/Elastic Search for logging.
● LightStep for tracing.
● Wavefront for stats.
27. Agenda
● Historical Lyft SoA architecture
● State of SoA networking in industry
● What is Envoy
● Envoy deployment @Lyft
● Envoy future directions
● Q&A
28. Envoy future directions
● Redis.
● More outlier detection and ejection (SR and latency).
● LB subset support.
● More rate limiting options / open source rate limit service / IP tagging.
● Configuration schema and better error output.
● Work with Google/community to add k8s support.
● Authentication and authorization.
● Envoy ecosystem?
29. Agenda
● Historical Lyft SoA architecture
● State of SoA networking in industry
● What is Envoy
● Envoy deployment @Lyft
● Envoy future directions
● Q&A
30. Q&A
● Thanks for coming!
● We are super excited about building a community around Envoy. Talk to us if
you need help getting started.
● https://lyft.github.io/envoy/
● Lyft is hiring: Contact us if you want to work on hard scaling problems in a fast
moving company: https://www.lyft.com/jobs