Minimize processing in the saga
Don’t share queues/processes across SLAs – decouple fault/surge isolation
Don’t use a saga when a normal handler will suffice
Rainbows, Unicorns, and other Fairy Tales in the Land of Serverless DreamsJosh Carlisle
When done correctly Serverless offers fantastic potential but can also lead to spectacular failure when critical concepts are overlooked. With over a dozen Serverless implementations on Azure Functions over the last couple years, I’ve learned some lessons the hard way. In this talk, I will be sharing a few of the most impactful hard-earned lessons and how I was able to overcome them. I’ll be touching on topics ranging from considerations using traditional relational databases, managing service and data connections to managing complexity and increasing observability. The talk is done in the context of Azure Functions but whose concepts apply equally to all Serverless Platforms.
One particular (and often forgotten) use-case for RavenDB is its usage as an embedded database. This operation mode allows application providers to abstract the complexity of database administration from their end-users while, at the same time, providing you a fully functional document store.
During this talk we will explore the challenges faced while deploying RavenDB in a massive number of machines throughout the globe (aiming at hundreds of thousands), and how RavenDB improved the capabilities of our application.
You ain't gonna need write a GenServer - Ulisses Almeida | Elixir Club UkraineElixir Club
Slides of Ulisses Almeida, Elixir Developer at The Coingaming Group, at Elixir Club Ukraine, Kyiv, 28.09.2019
Next conference - http://www.elixirkyiv.club/
Description of presentation
GenServers plays a central role in Elixir OTP applications.
However, when you overuse it, it can become an anti-pattern and introduce bottlenecks in your system.
In this talk you'll learn:
* What's a GenServer
* When GenServer can harm
* GenServer FOMO (fear of missing out)
* How Elixir ecosystem get you covered
After this talk, you'll understand how putting a GenServer in the wrong place can slowdown in your entire system.
You'll understand how the Elixir frameworks and libraries enable you to have reliable concurrency applications.
You'll feel no shame of being an Elixir developer and never had to write yourself a GenServer.
Follow us on social networks @ElixirClubUA and #ElixirClubUA
Announce and materials from conf - https://www.fb.me/ElixirClubUA
News - https://twitter.com/ElixirClubUA
Photo and free atmosphere - https://www.instagram.com/ElixirClubUA
*Organizer’s channel - https://t.me/incredevly
Overview of zone.js (Monkey Patching, Forking, Interception). How its used in Angular for triggering Change Detection and a demo in NodeJS for authentication handling.
All my slides and demos can be found here: https://github.com/IliaIdakiev/slides
Rainbows, Unicorns, and other Fairy Tales in the Land of Serverless DreamsJosh Carlisle
When done correctly Serverless offers fantastic potential but can also lead to spectacular failure when critical concepts are overlooked. With over a dozen Serverless implementations on Azure Functions over the last couple years, I’ve learned some lessons the hard way. In this talk, I will be sharing a few of the most impactful hard-earned lessons and how I was able to overcome them. I’ll be touching on topics ranging from considerations using traditional relational databases, managing service and data connections to managing complexity and increasing observability. The talk is done in the context of Azure Functions but whose concepts apply equally to all Serverless Platforms.
One particular (and often forgotten) use-case for RavenDB is its usage as an embedded database. This operation mode allows application providers to abstract the complexity of database administration from their end-users while, at the same time, providing you a fully functional document store.
During this talk we will explore the challenges faced while deploying RavenDB in a massive number of machines throughout the globe (aiming at hundreds of thousands), and how RavenDB improved the capabilities of our application.
You ain't gonna need write a GenServer - Ulisses Almeida | Elixir Club UkraineElixir Club
Slides of Ulisses Almeida, Elixir Developer at The Coingaming Group, at Elixir Club Ukraine, Kyiv, 28.09.2019
Next conference - http://www.elixirkyiv.club/
Description of presentation
GenServers plays a central role in Elixir OTP applications.
However, when you overuse it, it can become an anti-pattern and introduce bottlenecks in your system.
In this talk you'll learn:
* What's a GenServer
* When GenServer can harm
* GenServer FOMO (fear of missing out)
* How Elixir ecosystem get you covered
After this talk, you'll understand how putting a GenServer in the wrong place can slowdown in your entire system.
You'll understand how the Elixir frameworks and libraries enable you to have reliable concurrency applications.
You'll feel no shame of being an Elixir developer and never had to write yourself a GenServer.
Follow us on social networks @ElixirClubUA and #ElixirClubUA
Announce and materials from conf - https://www.fb.me/ElixirClubUA
News - https://twitter.com/ElixirClubUA
Photo and free atmosphere - https://www.instagram.com/ElixirClubUA
*Organizer’s channel - https://t.me/incredevly
Overview of zone.js (Monkey Patching, Forking, Interception). How its used in Angular for triggering Change Detection and a demo in NodeJS for authentication handling.
All my slides and demos can be found here: https://github.com/IliaIdakiev/slides
C* Summit 2013: Eventual Consistency != Hopeful Consistency by Christos Kalan...DataStax Academy
This session will address Cassandra's tunable consistency model and cover how developers and companies should adopt a more Optimistic Software Design model.
GRU: Taming a Herd of Wild Servers - Oz Katz, Similarweb - DevOpsDays Tel Avi...DevOpsDays Tel Aviv
"As SimilarWeb R&D grew, it has accumulated more and more physical servers, virtual machines and cloud instances.
Being production engineers, we regularly needed to locate servers when debugging issues, or when doing capacity planning. Trying to avoid adding another system to the mix, we managed to use our existing services, only adding a nice frontend for the humans.
In this talk I’ll present GRU, our lightweight, extensible inventory system. As part of this talk we will release GRU as a BSD 2 licensed open source project."
In the past few years, the terms serverless computing and microservices has been thrown around a lot, and it seems lots of people that use them do not understand what they are, and more importantly, what they aren't.
We'll try to provide some overview, along with a couple of definitions, a recap of the history, and speculation on the future - and of course, demo how Azure fits into the serverless story.
Queueing Theory is perhaps one of the most important mathematical theories in systems design and analysis, yet only few engineers learn it. This talk teaches the basics of queueing theory and explores the ramifications of queue behavior on system performance and resiliency. This talk aims to give practical skills that can be applied better build and tune your systems. The talk covers:
- Queueing delays
- Queueing capacity
- Little's Law and how to apply it
- Proper sizing of thread and connection pools
Experiences testing dev versions of MySQL and why it is good for youSimon J Mudd
Presentation given at OpenExpo Europe 2018 in Madrid on 6th June 2018
Each new version of MySQL comes out with exciting new features, many of which we’ve been asking for for a long time. The first development or DMR versions are released to the public some time before the software is considered production quality. So who is going to test these new versions which might break at any time and lose all your data?
booking.com does just this. The talk explains why we do it, what both we and the MySQL community gets out of it. If you’ve not considered doing such testing it’s very easy so come along and find out how. If you want to find out about some of the fun bugs we’ve seen then you’ll like this presentation too.
Database and Public Endpoints redundancy on Azure (CloudBrew)Radu Vunvulea
This is the perfect session if you need to design a solution that needs to run over Windows Azure and needs to be available 24 hours every day, 7 days per week. We will discuss different concepts and solutions to have databases and/or public API’s available even if a part of cloud infrastructure will be down. Topics like automatically backups, failover mechanism, traffic manager and redundancy will be discussed.
Building Distributed Systems with RavenDB NSCBcon NY by Oren Eini & Ayende Ra...Particular Software
RavenDB is:
No SQL Document Database
.NET / JVM / Node.JS / Python / REST / etc clients
High performance
Scalable & distributed
Safe by default
ACID
And those are just the highlights…
Eden Project Green Fingers Festival June 2014Shoot Gardening
Nicola Gammon (founder of shootgardening.co.uk) was guest speaker at the Eden Project in Cornwall during their Green Fingers Festival. Nicola talked about her own personal story which led her to launch Shoot and an overview of what Shoot offers members.
C* Summit 2013: Eventual Consistency != Hopeful Consistency by Christos Kalan...DataStax Academy
This session will address Cassandra's tunable consistency model and cover how developers and companies should adopt a more Optimistic Software Design model.
GRU: Taming a Herd of Wild Servers - Oz Katz, Similarweb - DevOpsDays Tel Avi...DevOpsDays Tel Aviv
"As SimilarWeb R&D grew, it has accumulated more and more physical servers, virtual machines and cloud instances.
Being production engineers, we regularly needed to locate servers when debugging issues, or when doing capacity planning. Trying to avoid adding another system to the mix, we managed to use our existing services, only adding a nice frontend for the humans.
In this talk I’ll present GRU, our lightweight, extensible inventory system. As part of this talk we will release GRU as a BSD 2 licensed open source project."
In the past few years, the terms serverless computing and microservices has been thrown around a lot, and it seems lots of people that use them do not understand what they are, and more importantly, what they aren't.
We'll try to provide some overview, along with a couple of definitions, a recap of the history, and speculation on the future - and of course, demo how Azure fits into the serverless story.
Queueing Theory is perhaps one of the most important mathematical theories in systems design and analysis, yet only few engineers learn it. This talk teaches the basics of queueing theory and explores the ramifications of queue behavior on system performance and resiliency. This talk aims to give practical skills that can be applied better build and tune your systems. The talk covers:
- Queueing delays
- Queueing capacity
- Little's Law and how to apply it
- Proper sizing of thread and connection pools
Experiences testing dev versions of MySQL and why it is good for youSimon J Mudd
Presentation given at OpenExpo Europe 2018 in Madrid on 6th June 2018
Each new version of MySQL comes out with exciting new features, many of which we’ve been asking for for a long time. The first development or DMR versions are released to the public some time before the software is considered production quality. So who is going to test these new versions which might break at any time and lose all your data?
booking.com does just this. The talk explains why we do it, what both we and the MySQL community gets out of it. If you’ve not considered doing such testing it’s very easy so come along and find out how. If you want to find out about some of the fun bugs we’ve seen then you’ll like this presentation too.
Database and Public Endpoints redundancy on Azure (CloudBrew)Radu Vunvulea
This is the perfect session if you need to design a solution that needs to run over Windows Azure and needs to be available 24 hours every day, 7 days per week. We will discuss different concepts and solutions to have databases and/or public API’s available even if a part of cloud infrastructure will be down. Topics like automatically backups, failover mechanism, traffic manager and redundancy will be discussed.
Building Distributed Systems with RavenDB NSCBcon NY by Oren Eini & Ayende Ra...Particular Software
RavenDB is:
No SQL Document Database
.NET / JVM / Node.JS / Python / REST / etc clients
High performance
Scalable & distributed
Safe by default
ACID
And those are just the highlights…
Eden Project Green Fingers Festival June 2014Shoot Gardening
Nicola Gammon (founder of shootgardening.co.uk) was guest speaker at the Eden Project in Cornwall during their Green Fingers Festival. Nicola talked about her own personal story which led her to launch Shoot and an overview of what Shoot offers members.
Daniel Marbach shows how to avoid common pitfalls in asynchronous code bases.
Learn how to:
* Differentiate between IO-bound vs CPU-bound work and how this relates to Threads and Tasks
* Avoid serious production bugs as a result of asynchronous methods returning void
* Opt-out from context capturing when necessary
* Deal with synchronous code in the context of asynchronous code
Cosilog, jeune société créee en septembre 2007, propose d'accompagner les établissements de santé autour de leurs projets d’organisation et de systèmes d'information via le conseil en stratégie et l’assistance à maîtrise d’ouvrage. Sa volonté est d’assurer un positionnement original et des solutions extrêmement concrètes.
After completing a POC typically we are left with a lot of open questions, most importantly how do we adopt this thing across the organization?
How do you begin to integrate the new message patterns and concepts into your legacy code bases? How will your development teams adopt the new tools and more importantly all the new concepts?
In this session, discover how using NServiceBus transforms the way we think about design and how we approach our work. Learn how typical three tiered designs just seem to disappear and how business processes start to become workflows.
See the recorded session here: https://particular-1.wistia.com/medias/t9lt8x1xyh
Connect front end to back end using SignalR and MessagingParticular Software
If you've ever worked on a message-based system, at some point you have probably asked the question: How can I connect my asynchronous back-end to the front-end?
This webinar is focused on answering this non-trivial question, outlining scenarios and solutions available in our toolbox to easily make the back-end talk to the front-end in a way that is both robust and scalable.
The cornerstone of the system we will study in this webinar is SignalR, a library that facilitates adding bi-directional communication between the server and the browser over the WebSocket protocol. We will also see how NServiceBus messaging framework can be used in combination with SignalR.
When the basic scenario is up and running, we will venture into making the system scale to many server instances. We will introduce the concept of a backplane that forwards messages to all server instances so that a client connected to one instance can receive messages sent from another one. We will implement the SignalR backplane using Redis, a popular open source in-memory key-value store.
Learn how to:
* Connect back-end systems to the frontend
* Use correlation, and why it's important
* Use SignalR with message-based systems
* Use SignalR with NServiceBus
* Achieve real time notifications in a JavaScript SPA
* Provide user feedback in web applications
Daniel Marbach showa how to combine Async/Await together with the Task Parallel Library to create a message pump for a service bus.
Learn how to:
* Deal with non-true asynchronous code paths
* Avoid unpleasant surprises when you combined Async/Await with the Task Parallel Library
* Achieve "graceful" shutdowns by applying cancellation to the asynchronous operations
* Achieve throttling with your concurrent operations without blocking unnecessarily
If you want to learn how a message pump built can be built with Async/Await and the Task Parallel Library looks like — don't miss this webinar!
Stored Procedure Superpowers: A Developer’s GuideVoltDB
Stored procedures often get a bad rap, but this webinar argues that many of the things people hate about stored procedures are more about the implementation, rather than the concept. Watch this webinar presented by John Hugg, Founding Engineer, VoltDB to understand how stored procedure “superpowers” can make your application faster, simpler and safer.
This is the story of how we managed to scale and improve Tappsi’s RoR RESTful API to handle our ever-growing load - told from different perspectives: infrastructure, data storage tuning, web server tuning, RoR optimization, monitoring and architecture design.
Slides from talk given on Java/Scala Lab 2014 in Odessa, Ukraine. Describes of how Java can be used as platform for latency-restricted applications such as High Frequency Trading and demonstrates how latencies 15-30µsec can be achieved on vanilla Oracle JDK.
Scaling a High Traffic Web Application: Our Journey from Java to PHP120bi
What makes an application scale? What should you worry about early on and what can wait?
Over the last 3 years, Achievers has learned many lessons and gained fundamental knowledge on scaling our SaaS platform. CTO Dr. Aris Zakinthinos will present and discuss the decisions we’ve made including language choice, server architecture, and much more; join us while we share tips, tricks, and things to absolutely avoid.
Throughout the evening you will have the opportunity to talk to the development team behind the Achievers Platform and ask questions on scaling best practices.
Everything fails all the time. Knowing how to deal with these failures in serverless applications becomes essential to building resilient, highly-available systems. In traditional monolithic applications, catching errors and handling retries is relatively straightforward. But as our systems become more distributed, we now have multiple (often asynchronous) components processing events from several sources, all with vastly different retry behaviors and failure mechanisms. Utilizing old patterns can cause errors to get swallowed, creating brittle, unreliable systems that are difficult to debug and hard to maintain.
In this talk, we’ll explore the built-in tools and processes that AWS has in place to appropriately deal with failures in distributed serverless applications.
Repeating History...On Purpose...with ElixirBarry Jones
A dive into the highlights of Elixir that make it the ideal platform for the web...and how all these questions were answered figured out 30 years ago. Presented to Upstate Elixir in Greenville, SC on Nov 16.
Scaling for Success: Lessons from handling peak loads on Azure with NServiceBusParticular Software
What happens when 200k users unexpectedly decide to use your platform simultaneously? We’re using autoscale on Azure PaaS so surely we can handle that, right? Wrong! Ask me how I found out… After going through a bit of trouble, I want to help you avoid the same mistakes I made.
Beyond simple benchmarks—a practical guide to optimizing code Particular Software
We know it’s vital that code executed at scale performs well. But how do we know if our performance optimizations actually make it faster? Fortunately, we have powerful tools which help—BenchmarkDotNet is a .NET library for benchmarking optimizations, with many simple examples to help get started.
In most systems, the code we need to optimize is rarely straightforward. It contains assumptions we need to discover before we even know what to improve. The code is hard to isolate. It has dependencies, which may or may not be relevant to optimization. And even when we’ve decided what to optimize, it’s hard to reliably benchmark the before and after. Only measurements can tell us if our changes actually make things faster. Without them, we could even make things slower, without realizing.
In this webinar you’ll learn how to:
- Identify areas of improvement which optimize the effort-to-value ratio
- Isolate code to make its performance measurable without extensive refactoring
- Apply the performance loop of measuring, changing and validating to ensure performance actually improves and nothing breaks
- Gradually become more “performance aware” without costing an arm and a leg
If there is one certainty in software, it's that things fail. It's not a matter of if but when. All too often, we throw the error at our users, who have no means of solving the problem except for trying again. Alternatively, we build custom code to address edge cases that can't easily be fixed, and we do so with a dangerous lack of insight into the problem at hand.
In this session, we'll discuss the importance of system resilience and how you can equip your software with the ability to recover from failure scenarios. After exploring different types of failures and considering different resilience strategies, we'll dig deeper into the retry pattern by rolling our own. We'll also see existing options, such as Polly and NServiceBus, that can handle this complexity for you.
Join me and embrace your system's failures.
“Have all my overdue invoices been paid?” Seems a simple enough question. But once you factor in the effects of time, even the simplest question can turn into a mess of edge cases and complicated batch jobs that never quite complete on time.
The NServiceBus Outbox gives you consistency between database and messaging operations, something that would be nontrivial to do on your own. So how does it work? And how can you prove that it works?
Serverless is the new hotness, but are Azure Functions right for your system?
Presented by Adam Jones, Chief Technology Officer for LHP Telematics, LLC based in Westfield, IN.
We know it's useful to split up complex systems. We've seen the benefits of modular deployment of microservices. Dealing with only one piece of code at a time eases our cognitive load. But how do we know where to draw the service boundaries? In complex business domains, it's often difficult to know where to start. When we get our boundaries wrong, the clocks starts ticking. Before long, we hear ourselves say "it would be easier to re-write it".
Join Adam for practical advice on discovering the hidden boundaries in your systems. Help tease out the natural separation of concerns in a sample business domain. During 20 years of developing complex systems, Adam has had plenty of time to get things wrong. Learn to avoid the common pitfalls that can lead us down the path to "the big re-write".
Webinar recording: https://particular.net/webinars/finding-your-service-boundaries-a-practical-guide
Monoliths are hard work. They're difficult to understand, brittle to change, time-consuming to test and risky to deploy. And you're stuck with the monolith's tech stack so you can't use any modern architectures or technologies.
Decomposing monoliths is easy if you take the right approach, and it results in a distributed solution with many small components. Those can be independently updated, tested and deployed, which means you release better software, more quickly.
You'll learn about:
- The right approach to decomposition
- the feature-driven approach, powered by Docker and NServiceBus
- How to run your monolith in a container
- How to extract features into new components
- How to plug components together with NServiceBus and run the whole stack in Docker containers
Daniel Marbach shows the differences between theory and practice when building a reliable message pump that consumes and produces messages from queues.
Trygve Lorentzen shows how he designs systems that the developers, support team, and decision makers at ProTeria can easily understand with the help of ServiceInsight.
William Brander and Sean Farmar show how the monitoring game changes when a system becomes distributed and you start delving into the world of microservices.
Learn:
* Why monitoring changes in distributed systems
* A monitoring philosophy that ensures all bases are covered
* The aspects of monitoring that affect asynchronous messaging systems
Making communications across boundaries simple with NServiceBusParticular Software
There are times when you should consider setting up secure communications between your software components across network boundaries.
Here are just a few:
* Your application is enormous (e.g., the global deployment of a marketing site targeting billions of people)
* Remoteness (e.g., your company has branch office locations around the globe)
* Your network constraints prevent communication (e.g., your machines in Azure Cloud Services are unable to talk to each other directly)
* You don't know the network conditions (e.g., IoT or mobile devices)
Yves Goeleven and Sean Feldman show how to overcome such challenges using NServiceBus.
Making communication across boundaries simple with Azure Service BusParticular Software
There are times when you should consider setting up secure communications between your software components across network boundaries.
Here are just a few:
* Your application is enormous (e.g., the global deployment of a marketing site targeting billions of people)
* Remoteness (e.g., your company has branch office locations around the globe)
* Your network constraints prevent communication (e.g., your machines in Azure Cloud Services are unable to talk to each other directly)
* You don't know the network conditions (e.g., IoT or mobile devices)
Yves Goeleven and Sean Feldman show how to overcome such challenges using Azure Service Bus.
There are many resources out there that walk you through the process of setting up distributed systems, queuing and asynchronous processes — with and without NServiceBus.
Despite all the online education, teams continue to make the same common mistakes when designing and implementing microservices architecture. While the mistakes can have devastating consequences, they are easy to avoid when approached intentionally.
Jeffrey Palermo and Justin Self share their experiences in overcoming common microservices pitfalls and show how NServiceBus naturally encourages better architecture, such as easy adherence to SOLID principles.
Learn:
* What a microservice really is (and is not)
* What mistakes teams commonly make
* How to avoid the pitfalls and design more robust and scalable architecture
* How to equip your team for a microservices architecture
A lot is changing in NServiceBus v6, especially with the changes required to support asynchronous message processing with Async/Await.
Daniel Marbach shows how the v6 API update helps to avoid common Async/Await pitfalls and makes your code ready for the asynchronous APIs in the cloud.
Learn how to:
* Leverage the new asynchronous APIs to avoid common Async/Await pitfalls
* Maximize the message throughput by using asynchronous APIs concurrently in your handlers
* Write synchronous code in the new asynchronous message handlers
* and last but not least, see how this relates to delicious Swiss Chocolate.
If you're an existing NServiceBus user and you want to be well prepared to migrate your solutions to the latest version of NServiceBus, or you're a complete newbie to NServiceBus and want to see how it can help you avoid common Async/Await pitfalls — don't miss this webinar!
When many developers think of Command-Query Responsibility Segregation (CQRS), they envision separate datastores for reads and writes, linked together by some kind of event-based synchronization mechanism. The reality is that for many domains this is overkill, while for others it still doesn’t solve fundamental concurrency issues.
How NServiceBus makes the entire process of scaling with asynchronous messaging easy, enabling you to focus on business logic, without forcing you to write a lot of infrastructure code.
Recording here: https://particular-1.wistia.com/medias/1vhxu4dqde
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
20 Comprehensive Checklist of Designing and Developing a WebsitePixlogix Infotech
Dive into the world of Website Designing and Developing with Pixlogix! Looking to create a stunning online presence? Look no further! Our comprehensive checklist covers everything you need to know to craft a website that stands out. From user-friendly design to seamless functionality, we've got you covered. Don't miss out on this invaluable resource! Check out our checklist now at Pixlogix and start your journey towards a captivating online presence today.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
7. We’ve screwed up!
• We’ve worked through all kinds of barriers to performance.
• We’ve employed NServiceBus on Dell.com, big retailers, financial
institutions, etc.
9. Executive summary
• Don’t distribute
• Martin Fowler’s First Law of Distributed Object Design: Don’t distribute your objects!
1. The network is reliable.
2. Latency is zero.
3. Bandwidth is infinite.
4. The network is secure.
5. Topology doesn't change.
6. There is one administrator.
7. Transport cost is zero.
8. The network is homogeneous.
• Minimize processing in the saga
• Don’t share queues/processes across SLAs – decouple fault/surge isolation
• Don’t use a saga when a normal handler will suffice
11. Problem: god Sagas
• All actions managed by the saga
• All work done in the context of the saga
• Very slow
• Hard to manage
• Let’s look at the code. . . RUN IT!!
12. god Sagas – test scenario
• Roughly 1000 orders created
• 16 minutes 26 seconds to complete
16. Solution
Use sagas only for orchestration – not integration or
processing logic
17. Refactor to: Thinner saga, single process
• Saga only manages taking and cancelling orders
• All handler methods refactored into classes
• More SOLID code (child’s play)
18. Remaining problem: single process
• Roughly 1000 orders created
• 9 minutes 57 seconds to complete
• Processed eventually!
• Let’s take a look at the code. . .
22. Refactoring: multiple processes
• Saga only manages taking and cancelling orders
• All handler classes refactored into their own processes
• Can scale handlers/processes independently
23. Refactored performance
• Roughly 1000 orders created
• 7 minutes 53 seconds to complete
•Processed twice as fast!
• Could be faster hosted across multiple machines
• Messages of varying throughput levels can maintain
SLAs
24.
25. A place for everything & everything in its place
26. Don’t overuse the saga. . .
• Unless you have an orchestration/workflow
• And get out of a saga method as fast as possible
• For maintaining any data other than orchestration (SagaData)
• Unless you have to
27. Come work with us in Austin!
Jeffrey Palermo
jeffrey@clear-measure.com
@jeffreypalermo
Andrew Siemer
andrew@clear-measure.com
@asiemer
#LoveClearMeasure