A Taxonomy of Clustering, or, No Container is an Island

•

2 likes•723 views

Covers the evolution from static deployments using individual Docker containers, to dynamic deployments in Kubernetes and Mesos, with a taxonomy of clustering, i.e., what's important about cluster managers. Talk given at the Software Development & Evolution Conference in Winnipeg, Manitoba, Canada on November 2nd, 2015.

Software

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Disclaimer

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Disclaimer #2

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
1. Applications with few pieces (includes
monoliths)
2. Service-oriented and applications with more
pieces – complex deployments
3. Need for Scaling and Resiliency
4. Sharing resources with other
services/applications (usually analytics)
5. How different tools and systems help with
each of these issues

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
SimpleApps

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
ApplicationWeb Server
Database
How to Deploy
Into Production?

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Single Machine
(maybe aVM)
Application
Database
Web Server

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
postgres-machine
my-app-machine
Application
Database
Web Server

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
my-app-machine
postgres-machine
nginx-machine
Application
Database
Web Server

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Logical Structure
my-app-machine
postgres-machine
nginx-machine
ApplicationWeb Server
Database

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Physical Structure:VMs
Infrastructure
Host Operating System
Hypervisor/Virtualization
bin/libs ruby bin/libs
nginx app postgres

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
VMs and Containers
Container Host OS
Infrastructure
Docker Engine (daemon)
bin/libs
ruby bin/libs
nginx
app postgres
Infrastructure
Host Operating System
Hypervisor/Virtualization
bin/libs ruby bin/libs
nginx app postgres

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Physical Structure: Containers
Container Host OS
Infrastructure
Docker Engine (daemon)
bin/libs
ruby bin/libs
nginx
app postgres

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Withmesofar?

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Deployment

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
application
postgres
nginx
Application
Database
Web Server
$ docker run -p 80:80 -p 443:443 -d nginx
cf9fb97bb4fadf673a58580d
$ docker run -d application
1238f2558ae2704d129cf
$ docker run -d postgres
4557c7c9d991238f2558ae27

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Static Deployment
application:
image: application:latest
links:
- postgres
postgres:
image: postgres:9.4
(docker-compose.yml)
nginx:
image: nginx:latest
links: apiservice
ports:
- "80:80"
- "443:443"
$ docker-compose up -d

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
application
postgres
nginx
Application
Database
Web Server

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
search cache
loggingapplication
postgres
nginx
Scaling: Add Caching

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Multiple Containers
application:
image: application:latest
links:
- logstash
- redis
- postgres
- elastic
nginx:
image: nginx:latest
links: apiservice
ports:
- "80:80"
- "443:443"
(docker-compose.yml)
logstash:
image: logstash:latest
links:
- apiservice
redis:
image: redis:latest
elastic:
image: elasticsearch:latest
postgres:
image: postgres:9.4
$ docker-compose up -d

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Scaling: MoreThan One?

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
load
balancer
Scaling: MoreThan One

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Where to PutThem?

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Availability: Dying Server

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Cluster Management

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Hi

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Dynamic
Deployment

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Scheduling Work

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Managing Resources

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Nouns

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Nodes

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Resources

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Workloads: Get Work Done

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
LabelingThings
node = FAST!

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Reservations

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Execution

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Executor

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Placement

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Scaling

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Reliability/Resilience

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
GettingConcrete

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
GoogleKubernetes
(Greek for "helmsman")

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Lessonsfrom
BorgandOmega

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
API service
log
forwarder
service
monitor
attached
storage
Pod: Unit of Placement

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Replication Controller

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
ApacheMesos

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
MesosDifferences

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Utilization

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Siloed Clusters

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Frameworks

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Scalability

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Resource Allocation

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
GreatTogether

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Explore, Play

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
Final Questions?

ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster
ThankYou!

What's hot

Software Supply Chain is a collective term used to describe the continuous integration and delivery pipelines. In addition, it refers to the observability tools that track what happens to a piece of code from the moment it’s in the source code to when it gets deployed, and everywhere in between. Grafeas is an open-source artifact metadata API to audit and govern your software supply chain. It's built as an industry standard for storing and retrieving metadata about software resources. Kritis is an open-source solution for securing your software supply chain for Kubernetes applications. It enforces deploy-time security policies using Grafeas. This talk will discuss the goals for each of the two open source projects, dive into the examples of how they can be used to secure your company's software supply chain, and conclude with the details of current and future development.

Software Supply Chain Management with Grafeas and Kritis

Aysylu Greenberg

Blast your app with gatling -

Padma Sudha

Practical webRTC - from API to Solution - webRTC Summit 2014 @ NYC

Alexandre Gouaillard

Spring IO '15 - Developing microservices, Spring Boot or Grails?

Fátima Casaú Pérez

Introduction to Mercurial

digitalspaghetti

Groovy in the Cloud

Daniel Woods

Software code review, i.e., the practice of having third-party team members critique changes to a software system, is a well-established best practice in both open source and proprietary software domains. Prior work has shown that the formal code inspections of the past tend to improve the quality of software delivered by students and small teams. However, the formal code inspection process mandates strict review criteria (e.g., in-person meetings and reviewer checklists) to ensure a base level of review quality, while the modern, lightweight code reviewing process does not. Although recent work explores the modern code review process qualitatively, little research quantitatively explores the relationship between properties of the modern code review process and software quality. Hence, in this paper, we study the relationship between software quality and: (1) code review coverage, i.e., the proportion of changes that have been code reviewed, and (2) code review participation, i.e., the degree of reviewer involvement in the code review process. Through a case study of the Qt, VTK, and ITK projects, we find that both code review coverage and participation share a significant link with software quality. Low code review coverage and participation are estimated to produce components with up to two and five additional post-release defects respectively. Our results empirically confirm the intuition that poorly reviewed code has a negative impact on software quality in large systems using modern reviewing tools.

The Impact of Code Review Coverage and Participation on Software Quality

Shane McIntosh

Git and GitHub - The beginning

Gemma Catolino

Software Supply Chain Management with Grafeas and Kritis

Aysylu Greenberg

Build systems orchestrate how human-readable source code is translated into executable programs. In a software project, source code changes can induce changes in the build system (aka. build co-changes). It is difficult for developers to identify when build co-changes are necessary due to the complexity of build systems. Prediction of build co-changes works well if there is a sufficient amount of training data to build a model. However, in practice, for new projects, there exists a limited number of changes. Using training data from other projects to predict the build co-changes in a new project can help improve the performance of the build co-change prediction. We refer to this problem as cross-project build co-change prediction. In this paper, we propose CroBuild, a novel cross-project build co-change prediction approach that iteratively learns new classifiers. CroBuild constructs an ensemble of classifiers by iteratively building classifiers and assigning them weights according to its prediction error rate. Given that only a small proportion of code changes are build co-changing, we also propose an imbalance-aware approach that learns a threshold boundary between those code changes that are build co-changing and those that are not in order to construct classifiers in each iteration. To examine the benefits of CroBuild, we perform experiments on 4 large datasets including Mozilla, Eclipse-core, Lucene, and Jazz, comprising a total of 50,884 changes. On average, across the 4 datasets, CroBuild achieves a F1-score of up to 0.408. We also compare CroBuild with other approaches such as a basic model, AdaBoost proposed by Freund et al., and TrAdaBoost proposed by Dai et al.. On average, across the 4 datasets, the CroBuild approach yields an improvement in F1-scores of 41.54%, 36.63%, and 36.97% over the basic model, AdaBoost, and TrAdaBoost, respectively.

Cross-Project Build Co-change Prediction

Shane McIntosh

Evolve your toolchains dev/ops with OpenStack

Ryan Richard

Landing code in curl

Daniel Stenberg

Simplified version for EKS (1.5 hr) Kubernetes 是目前雲端環境的顯學。可是，傳統的程式，並不是原封不動搬上去，就能夠自動享受 Kubernetes 所宣稱的種種好處。新的環境，不僅需要新的 Ops 思維，也需要新的 Dev 思維。我們將以一個半小時的時間，從軟體研發者的角度，探討軟體的設計該做哪些最起碼的改變，並體驗 Kubernetes 引進的新觀念及新效益。

給 RD 的 Kubernetes 初體驗 (EKS version)

William Yeh

During this presentation, I demonstrate how to implement authentication in your Java web applications using good ol' Java EE Container Managed Authentication, Spring Security and Apache Shiro. You'll also learn how to secure your REST API with OAuth and lock it down with SSL. After learning how to develop authentication, I'll introduce you to pentest your app, as well as OWASP, the OWASP Top 10, its Testing Guide and its Code Review Guide. Much of this talk is contained in demos and tutorials, which are available on my blog at http://raibledesigns.com/rd/tags/security and http://youtube.com/mraible.

Java Web Application Security - UberConf 2011

Matt Raible

Depending on your use cases you may need to access your databases which different patterns and technologies (CRUD+UI, batch, reactive, IoT, ...). At DataStax, the developer advocates team implements reference applications for developers. We had the chance to implement multiple approaches and can provide feedback. KillrVideo.com is one of this application, it has been written in 3 languages (Java, C# and Node) and implement API with REST, Grpc and GraphQL. Though live session, browsing real code, you will see implementation details, lessons learnt and get working source code in Github as takeaway.

Create API for your Databases

Cédrick Lunven

Microservice architectures are all the rage in JavaLand. They allow teams to develop services independently and deploy autonomously. Why microservices? IF you are developing a large/complex application AND you need to deliver it rapidly, frequently, and reliably over a long period of time THEN the Microservice Architecture is often a good choice Reactive architectures are becoming increasingly popular for organizations that need to do more, with less hardware. Reactive programming allows you to build systems that are resilient to high loads. In this session, I'll show you how to use JHipster to create a reactive microservices architecture with Spring Boot, Spring Cloud, Keycloak, and run it all in Docker. You will leave with the know-how to create your own resilient apps! Related blog post: https://developer.okta.com/blog/2021/01/20/reactive-java-microservices YouTube demo: https://youtu.be/clkEUHWT9-M YouTube recording: https://youtu.be/8OuZMFyh0xE GitHub repo: https://github.com/oktadev/java-microservices-examples/tree/main/reactive-jhipster

Reactive Java Microservices with Spring Boot and JHipster - Denver JUG 2021

Matt Raible

What's New in Spring for Apache Kafka 2.0

VMware Tanzu

Python+gradle

Stephen Holsapple

This presentation was held at the Spring One 2GX 2015 conference in Washington DC. The presentation explains how to migrate an existing Grails 2 application to the new Spring Boot and Gradle based Grails 3. It covers migrating plugins, applications and features gotchas as well as best praciteces. The conference presentation also included an extensive live coding section in which I migrated an existing application to Grails 3.

Migrating from Grails 2 to Grails 3

Michael Plöd

Abstract: So many platforms, so little time. How do we write an app for the web, Android, iOS, and other OS without spending all our time and money? Cross-platform can be your answer! You probably heard about React Native before, a framework that allows you to create native Android and iOS apps using React. But how do you get started with the framework? Whether you are an advanced iOS developer or a beginner React dev, this talk will help you get started with React Native. In this session, we explore React Native and its main concepts. Then, we will build a demo app from scratch for Android and iOS to show the power of cross-platform that the framework brings to the table! Link: https://dvinnik.dev/talks/mobile/hands-on-react-native/

Hands on React Native: From Zero to Hero

Dmitry Vinnik

What's hot (20)

Software Supply Chain Management with Grafeas and Kritis

Blast your app with gatling -

Practical webRTC - from API to Solution - webRTC Summit 2014 @ NYC

Spring IO '15 - Developing microservices, Spring Boot or Grails?

Introduction to Mercurial

Groovy in the Cloud

The Impact of Code Review Coverage and Participation on Software Quality

Git and GitHub - The beginning

Software Supply Chain Management with Grafeas and Kritis

Cross-Project Build Co-change Prediction

Evolve your toolchains dev/ops with OpenStack

Landing code in curl

給 RD 的 Kubernetes 初體驗 (EKS version)

Java Web Application Security - UberConf 2011

Create API for your Databases

Reactive Java Microservices with Spring Boot and JHipster - Denver JUG 2021

What's New in Spring for Apache Kafka 2.0

Python+gradle

Migrating from Grails 2 to Grails 3

Hands on React Native: From Zero to Hero

Similar to A Taxonomy of Clustering, or, No Container is an Island

Automated Java Deployments With Rpm

Martin Jackson

Today we are plagued by hundreds of choices when architecting a modern data center. Should our machines be virtual or physical? Should we use containers or Docker? Should we use a public cloud provider or a private cloud provider? Which configuration management tool is best to use? What about IaaS, PaaS, and SaaS? It would be manageable if these were binary choices; however, we often find ourselves in a hybrid environment.

OSDC 2017 | Taming the Modern Data Center by Seth Vargo

NETWAYS

OSDC 2017 - Seth Vargo - Taming the modern data center

NETWAYS

OSDC 2017 | Modern Secrets Management with Vault by Seth Vargo

NETWAYS

InfluxCloudi craft container orchestrator

Gianluca Arbezzano

OGCE Project Overview

marpierc

OGCE Overview for SciDAC 2009

marpierc

Not Only Streams for Akademia JLabs

Konrad Malawski

Are you a backend developer that’s being pushed into front end development? Are you frustrated with all JavaScript frameworks and build tools you have to learn to be a good UI developer? If so, this session is for you! We’ll explore the tools of the trade for frontend development (npm, yarn, Gulp, Webpack, Yeoman) and learn the basics of HTML, CSS, and JavaScript. This presentation dives into the intricacies of Bootstrap, Material Design, ES6, and TypeScript. Finally, after getting you up to speed with all this new tech, I'll show how it can all be found and integrated through the fine and dandy JHipster project.

Front End Development for Back End Developers - UberConf 2017

Matt Raible

Velocity London - Chaos Engineering Bootcamp

Ana Medina

State of Akka 2017 - The best is yet to come

Konrad Malawski

Microservices architecture has changed how companies develop and deploy applications. This change has affected testing process as well. New techniques have emerged and others have been enhanced. Does your organization wish to achieve unicorn like speed? With N deploys per day? Or a lead time (time to business value from idea to production) measured in days/weeks instead of months/years?In this session, we will not only describe how to use Istio and Kubernetes but actively demonstrate 3 specific techniques that you can use immediately following the session.

Sail In The Cloud

Alex Soto

Intro to Git Devnet-1080 Cisco Live 2018

Ashley Roach

Distributed Tracing

distributedtracing

Мы ведь тоже люди. Еретическая лекция про юзабилити инструментов разработчика

Nikita Prokopov

Fisl - Deployment

Fabio Akita

DevOps with Kubernetes and Helm - Jenkins World Edition

Jessica Deen

DevOps (Continuous Integrations, Continuous Delivery & Continuous deployment using Jenkins and Visual studio team services, setting up VTST build Agents, Integrating VSTS with SonarQube, NDepend,) , Complete automation of pushing code into VSTS from Visual Studio, Building Code by a Jenkin Server hosted on Azure and pushing that successful build on to Azure Web App via Release Pipeline or directly from Jenkins,VSTS Default agents, Setting up local agent from scratch, Setting up agents for code build, VSTS, Visual Studio Online Agents, Agent Pools, Hosted Agents, Hosted VS2017. Hosted Linux Agents, Setting up agent on VS Dev Test Labs, Setting up Template Parameters for Continuos Pipeline, Build Agent Creation Dynamically, Random Machine Name, Random Passwords, Dynamic Agent creation in VS Dev Test labs, Sonarcube, Code quality, Code Analysis, MSBuild, Integrate VSTS Build with NDepend, Package manager, Monolithic Architecture, Nuget, Package management, Npm js.com, Semantic versioning, Creating a nuget package, nuspec file, GitVersion Plugin, FeedURL, Chocolatey for package management, Chocolatey, chocolatey workflow,

Azure from scratch part 4

Girish Kalamati

Distributed tracing 101

Itiel Shwartz

使用 Prometheus 監控 Kubernetes Cluster

inwin stack

Similar to A Taxonomy of Clustering, or, No Container is an Island (20)

Automated Java Deployments With Rpm

OSDC 2017 | Taming the Modern Data Center by Seth Vargo

OSDC 2017 - Seth Vargo - Taming the modern data center

OSDC 2017 | Modern Secrets Management with Vault by Seth Vargo

InfluxCloudi craft container orchestrator

OGCE Project Overview

OGCE Overview for SciDAC 2009

Not Only Streams for Akademia JLabs

Front End Development for Back End Developers - UberConf 2017

Velocity London - Chaos Engineering Bootcamp

State of Akka 2017 - The best is yet to come

Sail In The Cloud

Intro to Git Devnet-1080 Cisco Live 2018

Distributed Tracing

Мы ведь тоже люди. Еретическая лекция про юзабилити инструментов разработчика

Fisl - Deployment

DevOps with Kubernetes and Helm - Jenkins World Edition

Azure from scratch part 4

Distributed tracing 101

使用 Prometheus 監控 Kubernetes Cluster

Recently uploaded

Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ? Venez le découvrir lors de cette session ignite

Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...

Anthony Dahanne

Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.

Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...

Shahin Sheidaei

Agnieszka Andrzejewska - BIM School Course in Kraków

bim.edu.pl

Petr Matuska, Sales & Sales Engineering Lead, GraphAware Western Australia Police Force’s adoption of Neo4j and the GraphAware Hume graph analytics platform marks a significant advancement in data-driven policing. Facing the challenges of growing volumes of valuable data scattered in disconnected silos, the organisation successfully implemented Neo4j database and Hume, consolidating data from various sources into a dynamic knowledge graph. The result was a connected view of intelligence, making it easier for analysts to solve crime faster. The partnership between Neo4j and GraphAware in this project demonstrates the transformative impact of graph technology on law enforcement’s ability to leverage growing volumes of valuable data to prevent crime and protect communities.

GraphAware - Transforming policing with graph-based intelligence analysis

Neo4j

Advanced Flow Concepts Every Developer Should Know

Peter Caitens

CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.

Cyaniclab : Software Development Agency Portfolio.pdf

Cyanic lab

Breaking the Code : A Guide to WhatsApp Business API.pdf

Meon Technology

WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation

WSO2

Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...

Abortion Clinic

AI/ML Infra Meetup May. 23, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Eric Wang (Software Engineer, @Uber) Uber has numerous deep learning models, most of which are highly complex with many layers and a vast number of features. Understanding how these models work is challenging and demands significant resources to experiment with various training algorithms and feature sets. With ML explainability, the ML team aims to bring transparency to these models, helping to clarify their predictions and behavior. This transparency also assists the operations and legal teams in explaining the reasons behind specific prediction outcomes. In this talk, Eric Wang will discuss the methods Uber used for explaining deep learning models and how we integrated these methods into the Uber AI Michelangelo ecosystem to support offline explaining.

AI/ML Infra Meetup | ML explainability in Michelangelo

Alluxio, Inc.

AI/ML Infra Meetup May. 23, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Junchen Jiang (Assistant Professor of Computer Science, @University of Chicago) Prefill in LLM inference is known to be resource-intensive, especially for long LLM inputs. While better scheduling can mitigate prefill’s impact, it would be fundamentally better to avoid (most of) prefill. This talk introduces our preliminary effort towards drastically minimizing prefill delay for LLM inputs that naturally reuse text chunks, such as in retrieval-augmented generation. While keeping the KV cache of all text chunks in memory is difficult, we show that it is possible to store them on cheaper yet slower storage. By improving the loading process of the reused KV caches, we can still significantly speed up prefill delay while maintaining the same generation quality.

AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG

Alluxio, Inc.

Data privacy is one of the most critical issues that businesses face. This presentation shares insights on the principles and best practices for ensuring the resilience and security of your workload. Drawing on a real-life project from the HR industry, the various challenges will be demonstrated: data protection, self-healing, business continuity, security, and transparency of data processing. This systematized approach allowed to create a secure AWS cloud infrastructure that not only met strict compliance rules but also exceeded the client's expectations.

Designing for Privacy in Amazon Web Services

KrzysztofKkol1

The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month. The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies. However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News. Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!

SOCRadar Research Team: Latest Activities of IntelBroker

SOCRadar

De mooiste recreatieve routes ontdekken met RouteYou en FME

Jelle | Nordend

How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?

XfilesPro

Crafting the Perfect Measurement Sheet with PLM Integration

Wave PLM

Unlocking Business Potential: Tailored Technology Solutions by Prosigns Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support. Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth. Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices. AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making. Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency. DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration. Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly. Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business. Join us on a journey of innovation and growth. Let's partner for success with Prosigns.

Prosigns: Transforming Business with Tailored Technology Solutions

Prosigns

Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...

rajkumar669520

Skilrock is an igaming technology platform & lottery software provider to the Global Lottery & Gaming Industry with a strong presence across continents. As part of the $2.4 billion SUGAL & DAMANI GROUP, Skilrock leverages the group's extensive domain experience to offer the INFINITI gaming platform - a true Omni-Channel, Omni-Gaming Platform that can serve any lottery or gaming operator anywhere in the world. INFINITI serves Retail, iGaming & Self Service Channels with equal ease and at the same time supports a wide variety of games like Lotto, Keno, Bingo, Instant, Sports, Poker, Rummy, Casino and Slots. Our popular solutions are Scratch Lottery, Retail Lottery, Instant Lottery and other igaming and lottery solutions.

iGaming Platform & Lottery Solutions by Skilrock

Skilrock Technologies

Key takeaways: Challenges of building platforms and the benefits of platformless. Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience. How Choreo enables the platformless experience. How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo. Demo of an end-to-end app built and deployed on Choreo.

Accelerate Enterprise Software Engineering with Platformless

WSO2

Recently uploaded (20)

Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...

Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...

Agnieszka Andrzejewska - BIM School Course in Kraków

GraphAware - Transforming policing with graph-based intelligence analysis

Advanced Flow Concepts Every Developer Should Know

Cyaniclab : Software Development Agency Portfolio.pdf

Breaking the Code : A Guide to WhatsApp Business API.pdf

WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation

Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...

AI/ML Infra Meetup | ML explainability in Michelangelo

AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG

Designing for Privacy in Amazon Web Services

SOCRadar Research Team: Latest Activities of IntelBroker

De mooiste recreatieve routes ontdekken met RouteYou en FME

How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?

Crafting the Perfect Measurement Sheet with PLM Integration

Prosigns: Transforming Business with Tailored Technology Solutions

Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...

iGaming Platform & Lottery Solutions by Skilrock

Accelerate Enterprise Software Engineering with Platformless

A Taxonomy of Clustering, or, No Container is an Island

1. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Ted M.Young @jitterted Sr. Software Engineer tedyoung@gmail.com Developer Productivity Group Guidewire Software, Inc.

2. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Ted M.Young @jitterted Sr. Software Engineer tedyoung@gmail.com Developer Productivity Group Guidewire Software, Inc.

3. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Disclaimer

4. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Disclaimer #2

5. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster

6. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster 1. Applications with few pieces (includes monoliths) 2. Service-oriented and applications with more pieces – complex deployments 3. Need for Scaling and Resiliency 4. Sharing resources with other services/applications (usually analytics) 5. How different tools and systems help with each of these issues

7. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster SimpleApps

8. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster ApplicationWeb Server Database How to Deploy Into Production?

9. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Single Machine (maybe aVM) Application Database Web Server

10. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster postgres-machine my-app-machine Application Database Web Server

11. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster my-app-machine postgres-machine nginx-machine Application Database Web Server

12. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Logical Structure my-app-machine postgres-machine nginx-machine ApplicationWeb Server Database

13. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Physical Structure:VMs Infrastructure Host Operating System Hypervisor/Virtualization bin/libs ruby bin/libs nginx app postgres

14. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster VMs and Containers Container Host OS Infrastructure Docker Engine (daemon) bin/libs ruby bin/libs nginx app postgres Infrastructure Host Operating System Hypervisor/Virtualization bin/libs ruby bin/libs nginx app postgres

15. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Physical Structure: Containers Container Host OS Infrastructure Docker Engine (daemon) bin/libs ruby bin/libs nginx app postgres

16. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Withmesofar?

17. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Deployment

18. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster application postgres nginx Application Database Web Server $ docker run -p 80:80 -p 443:443 -d nginx cf9fb97bb4fadf673a58580d $ docker run -d application 1238f2558ae2704d129cf $ docker run -d postgres 4557c7c9d991238f2558ae27

19. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Static Deployment application: image: application:latest links: - postgres postgres: image: postgres:9.4 (docker-compose.yml) nginx: image: nginx:latest links: apiservice ports: - "80:80" - "443:443" $ docker-compose up -d

20. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster application postgres nginx Application Database Web Server

21. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster search cache loggingapplication postgres nginx Scaling: Add Caching

22. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Multiple Containers application: image: application:latest links: - logstash - redis - postgres - elastic nginx: image: nginx:latest links: apiservice ports: - "80:80" - "443:443" (docker-compose.yml) logstash: image: logstash:latest links: - apiservice redis: image: redis:latest elastic: image: elasticsearch:latest postgres: image: postgres:9.4 $ docker-compose up -d

23. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Scaling: MoreThan One?

24. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster load balancer Scaling: MoreThan One

25. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Where to PutThem?

26. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Availability: Dying Server

27. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Cluster Management

28. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Hi

29. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Dynamic Deployment

30. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Scheduling Work

31. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Managing Resources

32. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Nouns

33. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Nodes

34. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Resources

35. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Workloads: Get Work Done

36. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster LabelingThings node = FAST!

37. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Reservations

38. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Execution

39. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Executor

40. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Placement

41. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Scaling

42. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Reliability/Resilience

43. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster GettingConcrete

44. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster GoogleKubernetes (Greek for "helmsman")

45. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Lessonsfrom BorgandOmega

46. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster API service log forwarder service monitor attached storage Pod: Unit of Placement

47. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Replication Controller

48. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Replication Controller

49. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Replication Controller

50. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Replication Controller

51. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster ApacheMesos

52. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster MesosDifferences

53. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Utilization

54. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Siloed Clusters

55. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Frameworks

56. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster

57. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster

58. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Scalability

59. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Resource Allocation

60. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster

61. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster GreatTogether

62. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster

63. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Explore, Play

64. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster Final Questions?

65. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster

66. ATaxonomy of Clustering @jitterted http://j.mp/sdec15cluster ThankYou!

A Taxonomy of Clustering, or, No Container is an Island

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to A Taxonomy of Clustering, or, No Container is an Island

Similar to A Taxonomy of Clustering, or, No Container is an Island (20)

More from Ted M. Young

More from Ted M. Young (6)

Recently uploaded

Recently uploaded (20)

A Taxonomy of Clustering, or, No Container is an Island