These are my summarized notes from all the microservices session I attended at QCon 2015. These sessions had tons of learning around how to scale microservices and avoid common pitfalls
Develop in ludicrous mode with azure serverlessLalit Kale
Today, every one of us wants to get things done fast. The fact of the matter is Serverless is a fantastic platform for doing things fast. Because, with Serverless, you really don’t have time to waste in terms of delivering your business value. Turns out you can with the right cloud services. In this talk we’ll create a microservice using Azure Functions and also get introduced to bigger picture of serverless computing.
I presented this session in Global Azure Bootcamp 2019 in Dublin. #GlobalAzure #AzureFunctions #Serverless
Discover and learn how to build a microservices platform, get a view of the best of breed architecture, solving common challenges, dig into Netflix stack, Yelp PaaSTA, AirBnB SmartStack, Apache Mesos, SoundCloud, Spinnaker experiences.
French audience : the JUG live recording is available here, https://www.youtube.com/watch?v=5LnL1HYmLwY&feature=youtu.be
Managing a Microservices Development Team (And advanced Microservice concerns)Steve Pember
So you’re decided to make the transition to a Microservice Architecture. You’ve spent time doing the research. You’ve designed out the responsibilities of each service with your team. You’ve read and memorized the entire article Martin Fowler wrote on the subject. Now, you’re running a team that’s tasked with building some Microservices. You’re built or extracted your first services. You’ve been successfully transmitting data between these services. What next? What should you be aware of? What should keep you up at night?
In this talk we’ll begin with a brief introduction to the architecture pattern before covering some of the more advanced topics when developing Microservices, focusing primarily on team management and service design philosophy. We’ll discuss the CAP theorem and why it should be your obsession. We’ll look at how Conway’s Law should be taken seriously and how it can serve as a warning to facilitate better communication between teams. Finally we’ll examine some common pitfalls of Microservices architectures and how they can be mitigated.
Exploring the drivers behind the Microservices hype, and defining the prerequisites in architecture and infrastructure needed before contemplating this path.
Presented at the first Sydney Microservices Meetup - Small Talk.
Develop in ludicrous mode with azure serverlessLalit Kale
Today, every one of us wants to get things done fast. The fact of the matter is Serverless is a fantastic platform for doing things fast. Because, with Serverless, you really don’t have time to waste in terms of delivering your business value. Turns out you can with the right cloud services. In this talk we’ll create a microservice using Azure Functions and also get introduced to bigger picture of serverless computing.
I presented this session in Global Azure Bootcamp 2019 in Dublin. #GlobalAzure #AzureFunctions #Serverless
Discover and learn how to build a microservices platform, get a view of the best of breed architecture, solving common challenges, dig into Netflix stack, Yelp PaaSTA, AirBnB SmartStack, Apache Mesos, SoundCloud, Spinnaker experiences.
French audience : the JUG live recording is available here, https://www.youtube.com/watch?v=5LnL1HYmLwY&feature=youtu.be
Managing a Microservices Development Team (And advanced Microservice concerns)Steve Pember
So you’re decided to make the transition to a Microservice Architecture. You’ve spent time doing the research. You’ve designed out the responsibilities of each service with your team. You’ve read and memorized the entire article Martin Fowler wrote on the subject. Now, you’re running a team that’s tasked with building some Microservices. You’re built or extracted your first services. You’ve been successfully transmitting data between these services. What next? What should you be aware of? What should keep you up at night?
In this talk we’ll begin with a brief introduction to the architecture pattern before covering some of the more advanced topics when developing Microservices, focusing primarily on team management and service design philosophy. We’ll discuss the CAP theorem and why it should be your obsession. We’ll look at how Conway’s Law should be taken seriously and how it can serve as a warning to facilitate better communication between teams. Finally we’ll examine some common pitfalls of Microservices architectures and how they can be mitigated.
Exploring the drivers behind the Microservices hype, and defining the prerequisites in architecture and infrastructure needed before contemplating this path.
Presented at the first Sydney Microservices Meetup - Small Talk.
Modern HA applications in nowadays are developed with set of small focused and discrete microservices. It's a trending concept and opens/solves questions like maintenance, scaling, live-deployments, security, fault-tolerance etc.
Making sense of microservices, service mesh, and serverlessChristian Posta
As companies move to become digital, we can get sidetracked and distracted by some of the changes in the technology landscape. Ideally we will be harnessing technology to solve the problems we have and leverage it to deliver software faster and safer. In this talk, I'll we'll take a look at some new technology trends in the open-source communities and when and how to use them.
Micro-services architecture is an evolutionary design ideal for evolutionary systems where you can’t fully anticipate the types of devices that may one day be accessing your application
This presentation is conducted on 14th Sept in Limerick DotNet User Group.
(https://www.meetup.com/preview/Limerick-DotNet/events/xskpdnywmbsb)
SlideShare Url: https://www.slideshare.net/lalitkale/introduction-to-microservices-80583928
In this presentation, new architectural style - Microservices and it's emergence is discussed. We will also briefly touch base on what are not microservices, Conway's law and organization design, Principles of microservices and service discovery mechanism and why it is necessary for microservices implementation.
About Speaker:
Lalit is a senior developer, software architect and consultant with more than 12 yrsof .NET experience. He loves to work with C# .NET and Azure platform services like App Services, Virtual Machines, Cortana, and Container Services. He is also the author of 'Building Microservices with .NET Core' (https://www.packtpub.com/web-development/building-microservices-net-core) book.
To know more and connect with Lalit, you can visit his LinkedIn profile below. https://www.linkedin.com/in/lalitkale/
This presentation will be useful for software architects/Managers, senior developers.
Do share your feedback in comments.
Lowering the risk of monolith to microservicesChristian Posta
Breaking down the monolith is a risky and difficult job. Is it even worth it? Can you improve your position with the monolith? If you do decide to decompose the monolith, how do you actually do it?
In this talk, we look at some of the difficult and nasty parts of decomposing the monolith and look at tools and techniques for lowering the risk for making these changes including tools like service mesh (Istio) and data management (Teiid spring boot, debezium.io, etc).
Follow along on twitter @christianposta or http://blog.christianposta.com for a write up and the videos from this talk.
How to Migrate to Cloud with Complete Confidence and TrustApcera
Henry Stapp, Director of Product Management at Apcera, explores the promises of the cloud and how new technologies (containers, micro-services, etc.) enable unparalleled speed and flexibility.
Early Draft: Service Mesh allows developers to focus on business logic while the crosscutting network data layer code is handled by the Service Mesh. This is a boon because this code can be tricky to implement and hard to test all of the edge cases. Service Mesh takes this a few steps further than AOP or Servlet Filters or custom language-specific frameworks because it works regardless of the underlying programming language being used which is great for polyglot development shops. Thus standardizing how these layers work, while allowing teams to pick the best tools or languages for the job at hand. Kubernetes and Istio Service Mesh automate best practices for DevSecOps needs like: failover, scale-out, scalability, health checks, circuit breakers, rate limiters, metrics, observability, avoiding cascading failure, disaster recovery, and traffic routing; supporting CI/CD and microservices architecture.
Istio’s ability to automate and maintaining zero trust networks is its most important feature. In the age of high-profile data breaches, security is paramount. Companies want to avoid major brand issues that impact the bottom line and shrink market capitalization in an instant. Istio allows a standard way to do mTLS and auto certificate rotation which helps prevent a breach and limits the blast radius if a breach occurs. Istio also takes the concern of mTLS from microservices deployments and makes it easy to use taking the burden off of application developers.
Your Journey to Cloud-Native Begins with DevOps, Microservices, and ContainersAtlassian
Everyone is excited about cloud-native applications. And for good reason! They're scalable, resilient, portable across cloud environments, and make it easier to incorporate customer feedback quickly. But there's a catch: cloud-native applications fundamentally change the way you provision, deploy, and manage your infrastructure.
That's where DevOps, microservices, and containers come in. This session will show you how to combine them to create a highly-automated continuous delivery platform. By streamlining the process to resemble factory assembly lines, you can adapt quickly to market changes and keep your customers happy – without burning your team out.
The introduction covers the following
1. What are Microservices and why should be use this paradigm?
2. 12 factor apps and how Microservices make it easier to create them
3. Characteristics of Microservices
Note: Please download the slides to view animations.
A high-level overview of Microservices architecture topics you should be familiar with before you actually start breaking your monolith into microservices
FOR THE FULL VIDEO, RECORDING & PRESENTATION:
https://typesafe.com/blog/going-reactive-in-java-with-typesafe-reactive-platform
--
In this presentation by Jamie Allen, we do a deep dive into the Typesafe Reactive Platform from the Java developer’s perspective, to learn how Typesafe supports the entire Reactive application development lifecycle.
Reactive application development is becoming mainstream and considered a mission-critical need for future growth. This new wave of business applications are message-driven, elastic, resilient and responsive by nature, designed to scale elastically and maintain responsiveness during even large failures. With the Typesafe Reactive Platform (RP), including Play Framework and Akka, Java developers can start to use tools designed for building distributed systems that deliver highly-responsive user experiences. Regardless of whether you code in Java or Scala, Typesafe RP provides a resilient and message-driven application stack that scales effortlessly on multicore and cloud computing architectures.
A survey of problems involved in building containers and build tools such as:
buildah
nixos-container
ansible-container
Smith
Distroless
Buildkit
Source to Image (s2i)
Habitat
This is a small introduction to microservices. you can find the differences between microservices and monolithic applications. You will find the pros and cons of microservices. you will also find the challenges (Business/ technical) that you may face while implementing microservices.
Modern HA applications in nowadays are developed with set of small focused and discrete microservices. It's a trending concept and opens/solves questions like maintenance, scaling, live-deployments, security, fault-tolerance etc.
Making sense of microservices, service mesh, and serverlessChristian Posta
As companies move to become digital, we can get sidetracked and distracted by some of the changes in the technology landscape. Ideally we will be harnessing technology to solve the problems we have and leverage it to deliver software faster and safer. In this talk, I'll we'll take a look at some new technology trends in the open-source communities and when and how to use them.
Micro-services architecture is an evolutionary design ideal for evolutionary systems where you can’t fully anticipate the types of devices that may one day be accessing your application
This presentation is conducted on 14th Sept in Limerick DotNet User Group.
(https://www.meetup.com/preview/Limerick-DotNet/events/xskpdnywmbsb)
SlideShare Url: https://www.slideshare.net/lalitkale/introduction-to-microservices-80583928
In this presentation, new architectural style - Microservices and it's emergence is discussed. We will also briefly touch base on what are not microservices, Conway's law and organization design, Principles of microservices and service discovery mechanism and why it is necessary for microservices implementation.
About Speaker:
Lalit is a senior developer, software architect and consultant with more than 12 yrsof .NET experience. He loves to work with C# .NET and Azure platform services like App Services, Virtual Machines, Cortana, and Container Services. He is also the author of 'Building Microservices with .NET Core' (https://www.packtpub.com/web-development/building-microservices-net-core) book.
To know more and connect with Lalit, you can visit his LinkedIn profile below. https://www.linkedin.com/in/lalitkale/
This presentation will be useful for software architects/Managers, senior developers.
Do share your feedback in comments.
Lowering the risk of monolith to microservicesChristian Posta
Breaking down the monolith is a risky and difficult job. Is it even worth it? Can you improve your position with the monolith? If you do decide to decompose the monolith, how do you actually do it?
In this talk, we look at some of the difficult and nasty parts of decomposing the monolith and look at tools and techniques for lowering the risk for making these changes including tools like service mesh (Istio) and data management (Teiid spring boot, debezium.io, etc).
Follow along on twitter @christianposta or http://blog.christianposta.com for a write up and the videos from this talk.
How to Migrate to Cloud with Complete Confidence and TrustApcera
Henry Stapp, Director of Product Management at Apcera, explores the promises of the cloud and how new technologies (containers, micro-services, etc.) enable unparalleled speed and flexibility.
Early Draft: Service Mesh allows developers to focus on business logic while the crosscutting network data layer code is handled by the Service Mesh. This is a boon because this code can be tricky to implement and hard to test all of the edge cases. Service Mesh takes this a few steps further than AOP or Servlet Filters or custom language-specific frameworks because it works regardless of the underlying programming language being used which is great for polyglot development shops. Thus standardizing how these layers work, while allowing teams to pick the best tools or languages for the job at hand. Kubernetes and Istio Service Mesh automate best practices for DevSecOps needs like: failover, scale-out, scalability, health checks, circuit breakers, rate limiters, metrics, observability, avoiding cascading failure, disaster recovery, and traffic routing; supporting CI/CD and microservices architecture.
Istio’s ability to automate and maintaining zero trust networks is its most important feature. In the age of high-profile data breaches, security is paramount. Companies want to avoid major brand issues that impact the bottom line and shrink market capitalization in an instant. Istio allows a standard way to do mTLS and auto certificate rotation which helps prevent a breach and limits the blast radius if a breach occurs. Istio also takes the concern of mTLS from microservices deployments and makes it easy to use taking the burden off of application developers.
Your Journey to Cloud-Native Begins with DevOps, Microservices, and ContainersAtlassian
Everyone is excited about cloud-native applications. And for good reason! They're scalable, resilient, portable across cloud environments, and make it easier to incorporate customer feedback quickly. But there's a catch: cloud-native applications fundamentally change the way you provision, deploy, and manage your infrastructure.
That's where DevOps, microservices, and containers come in. This session will show you how to combine them to create a highly-automated continuous delivery platform. By streamlining the process to resemble factory assembly lines, you can adapt quickly to market changes and keep your customers happy – without burning your team out.
The introduction covers the following
1. What are Microservices and why should be use this paradigm?
2. 12 factor apps and how Microservices make it easier to create them
3. Characteristics of Microservices
Note: Please download the slides to view animations.
A high-level overview of Microservices architecture topics you should be familiar with before you actually start breaking your monolith into microservices
FOR THE FULL VIDEO, RECORDING & PRESENTATION:
https://typesafe.com/blog/going-reactive-in-java-with-typesafe-reactive-platform
--
In this presentation by Jamie Allen, we do a deep dive into the Typesafe Reactive Platform from the Java developer’s perspective, to learn how Typesafe supports the entire Reactive application development lifecycle.
Reactive application development is becoming mainstream and considered a mission-critical need for future growth. This new wave of business applications are message-driven, elastic, resilient and responsive by nature, designed to scale elastically and maintain responsiveness during even large failures. With the Typesafe Reactive Platform (RP), including Play Framework and Akka, Java developers can start to use tools designed for building distributed systems that deliver highly-responsive user experiences. Regardless of whether you code in Java or Scala, Typesafe RP provides a resilient and message-driven application stack that scales effortlessly on multicore and cloud computing architectures.
A survey of problems involved in building containers and build tools such as:
buildah
nixos-container
ansible-container
Smith
Distroless
Buildkit
Source to Image (s2i)
Habitat
This is a small introduction to microservices. you can find the differences between microservices and monolithic applications. You will find the pros and cons of microservices. you will also find the challenges (Business/ technical) that you may face while implementing microservices.
The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.
In this slide we have discussed, Monolithic application vs Microservices, applicable scenarios for adopting the architectural pattern, when we need microservices, what are the benefits, case study of an e-commerce platform by compartmentalizing the scopes into different sample microservices and Docker implementations.
The full talk has been recorded here: https://youtu.be/tNlp7HS533g
Whar are microservices and microservices architecture (MSA) How we reach them? Are they the same or SoA or not? When to use them? What are the key characteristics?
Slides of my talk given in #Gapand2017 in Andorra
“ The Microservices architecture has many appealing qualities, but the road towards it has painful traps for the unwary. This book will help you figure out if this path is for you, and how to avoid those traps on your journey.”
—Martin Fowler Chief Scientist, ThoughtWorks
Presentazione dello speech tenuto da Carmine Spagnuolo (Postdoctoral Research Fellow - Università degli Studi di Salerno/ ACT OR) dal titolo "Technology insights: Decision Science Platform", durante il Decision Science Forum 2019, il più importante evento italiano sulla Scienza delle Decisioni.
A journey of a FinTech application into the cloud, starting as a monolith and growing to MicroServices. Why going functional is so good for finance technology, and how Scala helped us build a better application.
Where SOA and Monolitch EAR have failed. It's not simple to have your Apps scaling automagically without a very complex architecture. We're going to show pros and cons of so called Cloud-Native Applications based on Microservices, Caas, DevOps, Continuous Delivery....
This talk was done in Feb 2020. Sergey and I co-presented at CTO Forum on Microservices and Service Mesh (how they relate, requirements, goals, best practices and how DevOps and Agile has had convergence in the set of features for Service Mesh and gateways around observability, feature flags, etc.)
Comparing and contrasting monolithic systems to Lego pieces (microservices) at the 50,000 foot view. In this presentation we will compare and contrast monolithic systems to microservices. We will then take a look at some of the down sides to microservices. And then we will discuss some strategies for building microservices.
Istio as an enabler for migrating to microservices (edition 2022)Ahmed Misbah
This session is targeted towards teams and organizations considering to migrate their applications from monolithic to Microservice architecture by proposing Istio as an enabler. Istio is an implementation of service mesh, a technology useful for migrating to Microservices iteratively and safely.
Migrating application architectures to Microservices is considered a key area of transformation in the IT world. Modernizing legacy applications to Kubernetes-based Microservices can prove to be very challenging if not planned correctly, taking into consideration the right technologies and enablers.
This session explains how Istio can be used as a bridge and enabler for modernizing legacy monolithic applications to Microservices. Topics covered in the session will include:
1- Advantages of migrating to Microservices and service mesh .
2- Designing a Microservice application based on splitting an existing monolithic application.
3- Implementing Microservices iteratively as a strangler fig application with Istio.
4- Features Istio provides as a service mesh platform.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
3. KeyNote: Micro Chips to
Microservices
• 1945 - 2015 - We have increased the calc/sec by 1 Trillion
• Hardware mainly scaled by miniaturization & abstraction like plug n play.
• Software doesn’t really scale by abstraction e.g. the software we write today hasn’t
abstracted much after high level language e.g. fortran and HTML are in the same
generation.
• Software scales by federation and wide participation
• Wide participation means lots of different people bring their knowledge to find out different
things and contribute to knowledge base.
• Federated architecture means you can do stuff like create your own website, app and you
can do whatever you want individually without effecting other parts of internet or app
ecosystem and then bring your learning and share with rest of the world.
• So if you are thinking of scaling something think about how can you federate and then
share the learnings from each federation.
4. KeyNote: Micro Chips to
Microservices
One of the ways to do this federated architecture is Micro-
services and here are few points about it:
1. Has to be independently Deployable (fundamental value)
2. End-to-end responsibility of one single small team
3. No Central Database
4. Extensive Automation & Monitoring
5. Smart Versioning Services
5. KeyNote: Micro Chips to
Microservices
Checklist before you start embarking on Micro-services track
1. Is it right for the domain? (usually very high volumes)
2. Do you understand the domain boundaries? (refactoring
across services is very hard)
3. Can you maintain strict discipline? (interaction restricted to
hardened interfaces)
4. Can you have high situational awareness about your
systems? i.e. consumer knows that producer before
deployment that interfaces are same not only relying on
Mocks
6. KeyNote: Micro Chips to
Microservices
Strategies for Monolith
• Start packing related code into a container
• For complex system, large releases can cause lot of
instability. So you should do small continuous
deployment
• Deployment is different from releases, switching
feature flags on and off is considered releasing
8. Netflix’s Viewing Data Microservices
• Netflix Scale
• 62 Million members
• 50+ counteries
• 3 Billion Hours/Month
• 1000+ device types
• 37% Downstream bandwidth of N/A
9. Netflix’s Viewing Data Microservices
• Viewing Data Services need to calculate
• Who, What, When, Where (physical and device) and how long they
watched
• How does this benefits users?
• How does this benefit Users?
• Pickup where you left off (switch devices)
• MyActivity timeline
• Helps Netflix see the quality like what was buffer speed etc.
• Helps them subscription purchase e.g. they figured out that Adam
Sandler does well in all the regions, so they made a deal with him
10. Netflix’s Viewing Data Microservices
High volume events like Hearbeats events and Read Last events both going
to stateful tier
System Architecture before Microservices
13. Netflix’s Viewing Data Microservices
• Why change? (assume all this risk and cost)
• Current System would have worked really great for next few years,
monolith are not a bad thing and many times its actually great
• They wanted to do before the imminent need so they can do
mindfully
• They wanted to work in a mode where there is no maintenance
mode
• Rapid growth due to Virtuous Cycle: Viewing - Improved
Personalization - Better Experience - more viewing
• Stateful instance count remain same 24/7 regardless of loads
14. Netflix’s Viewing Data Microservices
• How did we do that?
• Shadow Testing
• All the request goes through both the systems legacy and micro
services but only the legacy system serve the users.
• This helps not only making sure that its operating correctly but also that
its working properly at scale
• Traffic Dial
• To do this they needed to make sure they have a consistsnt view of the
world. So they sacrificed bit of pure microservice system and removed
persistent from services and pointed them to old legacy system
• 1% of the traffic was directly hitting some of the service then dial up
from there to 100%
15. Netflix’s Viewing Data Microservices
Key Points
• Devour the whale a bite at a time
• Design for idempotency so it can replayed (using
something like CQRS/Event Sourcing)
• System architectures are throw-away artefacts,
useful for only a limited time. Design for 10X plan
to rewrite before 100x - System Arc is a throw
away artifact.
17. Engineering for Scale at VMTurbo
• Who are VM Turbo?
• VMTurbo is a data center control system and does automatic
resource allocation
• They do that by creating marketplace between workload
(applications, VMs, containers) as buyers and CPU, storage, fabric
as sellers
• Why not architecture from microservices from beginning?
• Monolith allows you to explore the complexity of a system and its
component boundaries
• Martin fowler said in a recent article how you shouldn’t start with
microservices in the beginning to get understanding of your domain
19. Engineering for Scale at VMTurbo
• Problems
• Release cycle for 6 months with interim patches
• No metrics captured
• Monolothic team because of monolithic architecture
• Monolothic Architecture caused scalability, concurrency and tangled interface
between components.
• Catalysts for Change
• Growth in customer base
• More Large Environments
• Geographical Spread of Team
• More Frequent Deleviries
20. Engineering for Scale at VMTurbo
• Steps to create first service
• Clean up the interface between Mediation (service to
be created) and the Analysis component
• Separate Mediation component completely from
Monolith
• Publish APIs - so anyone can write mediation
component and works with VMTurbo Analysis
21. Microservices and the art of
taming the dependency hell
Michael Bryzek Cofounder & ex-CTO Gilt
@mbryzek
mbryzek@alum.mit.edu
22. Microservices and the art of taming
the dependency hell
• Gilt which is an organization with 1500 git repos and
over 150 micorservices. Its about 1000 people with
150+ people in tech
• Gilt started with very basic Rails monolithic
architecture
• They started with excessive caching but soon they
realize that its very hard to reason about how adding
any new feature will effect in realtime.
• They started minimizing caching but depending on
very fast data store
23. Microservices and the art of taming
the dependency hell
• They started extracting the highly available parts of the application
and put them behind service. Few services they extracted were:
• User Service with 10K RPS with millions of users. They went for
mongoDB to give absolute consistency instead of eventual
consistency.
• Catalog Service with 5K RPS and used relational DB
• Inventory Service with 10K RPS+ with guarantee never oversold
and used HBase.
• Cart Service with low throughput and used Dynamo DB
• Any significant features they add start becoming the service.
24. Microservices and the art of taming
the dependency hell
• What are the problems once they started adding services?
• Builds get larger and slower - You keep depending on services and
start adding client libraries for services now you have world
downloaded
• Create new client libraries that are each just a little bit different
• Produce custom APIs instead of consistent APIs that reduce
interoperability
• Increase amount of boilerplate code
• Reduce code quality; slow down development
• And Eventually you will see a production error
25. Microservices and the art of taming
the dependency hell
• To minimize all those pains they followed guiding principle
called The Open Source Way.
• This means making the way we build propriety software
similar to the way open source is done. In the following
areas:
• How do they provide documentation?
• How does the library integrates with other apps?
• How do i get support/contribute/report bugs?
• public or private is a detail.
26. Microservices and the art of taming
the dependency hell
Some specific strategies to avoid these problems and manage dependencies are
1. Tooling Matters
• Anyone who has succeeded with Microservices is they have used extensive
tooling to automate stuff.
2. API Design must be first class
• Design of your API and the data structures are hardest to change
afterwards.
• You can tools like Protobufs, thrift, avro, swagger 2.0 and apidoc to make
schema as your first class and makes it very easy for consumer to knows
what data its getting.
• Schema first design is the most important concept to avoid the dependency
hell.
27. Microservices and the art of taming
the dependency hell
3. Accurate Documentation
• Documentation should be of the similar to the amount needed for an open
source project to be successful.
• Using semantic versioning to point out if there is any breaking changes.
• Accurate documentation can be achieved by producing the documentation in
software process
4. Generating Client Side Libraries
• Makes it easy to test
• Reduces lot of boilerplate
• Consistent naming
• You can minimize the external dependencies
28. Microservices and the art of taming
the dependency hell
5. Backward Compatibility
• Renaming just doesn’t work.
• Introduce new model, migrate and deprecate all the old stuff
6. Forward Compatibility
• Your service shouldn’t blow up if new field is added or seen by
the system.
• Careful of enums, what happens when you add a value in the
future.
• Don’t throw exception if new field shows up
29. Scaling Stack Overflow: Keeping it
Vertical by Obsessing Over Performance
(case for monoliths)
David Fullerton
@df07
30. Scaling Stack Overflow: Keeping it Vertical by Obsessing
Over Performance
• 4 Billion Requests/Month, 3K requests/s, 45 M uniques/month, 8K qs/month
and 500,000 page-views/month
• 34 Devs, 6 sysadmins, 6 designers, 75% remote
• Their Architecture
• 2 HAProxy - one failover
• 9 web servers
• 4 SQL Servers (Vertical scaling - 2 clusters)
• 2 Redis Servers
• 3 Elastic Servers
• 3 Tag Engine
32. Scaling Stack Overflow: Keeping it Vertical by Obsessing
Over Performance
• It’s what they called Monolith Plus most of the stuff happens in web
tier and DB.
• It scales really good for them.
• All of their web servers and SQL servers are running under 10% CPU
consumption and majority of the RAM consumption is under 70%
• Deploys all day everyday, deploy through web tier in 3 minutes. This
gives them huge ability to test on production since they can roll out so
fast
• Testing on users, few unit test and integration test not lot of
automated tests.
• Big believer in feature flags.
33. Scaling Stack Overflow: Keeping it Vertical by Obsessing
Over Performance
!
• This works specifically for stackoverflow because
• Read-heavy load centered on one page.
• Forgiving community — they released the bug with alert on the homepage
• How do they work?
• Start with what they know
• Measure it live
• Fix the slow - because performance is a feature
• Use excessive caching
• Optimizing for performance instead of scaling out
34. Scaling Stack Overflow: Keeping it
Vertical by Obsessing Over Performance
“my primary guideline would be don’t even consider
micro service unless you have a system that’s too
complex to manage as a monolith” — Martin Fowler
35. The Seven Deadly Sins of
Microservices
Daniel Bryant
@danielbryantuk
36. Seven Deadly Sins of Microservices
1. Lust - using the latest and greatest
• Choose Boring Technology
• Use Matt Raibel’s comparison framework to add objectivity
2. Gluttony - Excessive communication protocols
• Choose initially only one Sync (e.g. JSON over HTTP) and one Async ( eg. RabbitMQ) protocol
3. Greed - All your Service are belong to us
• Don’t underestimate the effect it will happen on your organization not necessarily only technical
side
• Few good books - The Connected Company, The Modern Firm, On Chnage Management
4. Sloth - Creating a distributed monolith
• If you can’t deploy services independently then you are not doing micro services
37. Seven Deadly Sins of Microservices
5. Wrath - Blowing up when bad things happen
• Putting the chaos monkey is really useful to put in your deployment pipeline
• Read up on Release It!
6. Envy - The shared single domain fallacy
• One model breaks encapsulation and introduces coupiing
• Know DDD
7. Pride - Testing the world of transience
• There is a mindset change in how you are testing
• Invest in your Build Pipeline
• Use Serenity BDD
• Wiremock - testing in jenkins fault tolerence
• Testing in production? - Netflix and Gilt - once you reach at certain level of services only way to test is in
production
38. Summary
• Know your domains very well before you start creating microservices because refactoring
across services is very hard
• You are not doing microservice if you are not independently deploying
• Make API design first class by using tools like swagger and apidoc
• Architect for failure and build the failure testing in your build pipeline by using tools like
Wiremock
• Testing is really hard because its impossible to have the full view of the system all the time. So
have to invest in tools, API documentation and build pipeline to better gauge your system.
• Use generated clients to avoid tons of boilerplate
• Don’t borrow other people problems, figure out your own pain points.
• Microservices have a hefty tax and it usually worth it if the team size, complexity or the scale
of the app is growing
• Monoloiths maybe the best architecture in certain domains eg. Stackoverflow