SlideShare a Scribd company logo
1 of 26
Scalable Service Architectures
Lessons learned
Zoltán Németh
Sr Engineering Manager, Streaming & PlayerAn IBM company
 Founded in 2007, acquired by IBM in 2016
 Live streaming and VOD
 Freemium / Pro / Demand / Align
An IBM company
Streaming flow
Now back to 2009...
Earthquake in Japan
Protests in Ukraine, Egypt, Syria
Asteroid Approach
SpaceX Launch
El Classico
We must scale
Defining scalability
Scalability is the ability to handle increased workload
by repeatedly applying a costeffective strategy for
extending a system’s capacity.
(CMU paper, 2006)
How well a solution to some problem will work when
the size of the problem increases. When the size
decreases, the solution must fit. (dictionary.com and
Theo Schlossnagle, 2006)
Self-contained
service
 Explicitly declare and
isolate dependencies
 Isolation from the outside
system
 Static linking
 Pay attention to GPL
 Do not rely on system
packages
Disposability  Maximize robustness with
fast startup and graceful
shutdown
 Disposable processes
 Graceful shutdown on
SIGTERM
 Handling sudden death:
robust queue backend
Backing Services  Treat backing services as
attached resources
 No distinction between
local and third party
services
 Easily swap out resources
 Export services via port
binding
 Become the backing
service for another app
Drawing source: 12factor.net
Processes,
concurrency
 Stateless processes (not
even sticky sessions)
 Process types by work type
 We <3 linux process
 Container > VM
 Shared-nothing  adding
concurrency is safe
 Process distribution
spanning machines
Statelessness  Store everything in a
datastore
 Aggregate data
 Aggregator / map &
reduce
 CQEngine
 Chandra
 Scalable datastores
 Handling user sessions
Microservices  Self-contained
 Disposable
 Stateless
 Shared-nothing
 API communication
 Dependency management
moved to external
 Be Warned!
Image credits: christofcoetzee.co.za, techblog.netflix.com
Monitoring  Metrics collecting
 Graphite, New Relic
 Self-aware applications
 Cluster state
 Zookeeper, Consul
 Scaling decisions
 Capacity amount
 Graph derivative
 App requests
Load Balance  DNS or API
 App level balance
 Uniform entry point or
proxy
 Balance decisions
 Load
 Zookeeper state
 Resource policies
Service
Separation
 Rate limiting
 Failure is inevitable
 Circuit Breaker pattern
 Stop cascading failure,
allow recovery
 Fail fast, fail silent
 Hystrix
 Service decoupling
 Asynchronous operations
Deployment  Automate all the things
 Chef & VMs
 Docker
 Immutable deployment
 Docker / Kubernetes /
Rancher
 Handling tasks before
shutdown
Extras  Debugging features
 Log processing:
Logstash, Kibana
 Clojure / JS consoles
 Runtime configuration
via env
 Scaling API
 Cloud providers
 Automatic start / stop
Reading
 Scalable Internet Architectures by Theo Schlossnagle
 The 12-factor App: http://12factor.net/
 Carnegie Mellon Paper: http://www.sei.cmu.edu/reports/06tn012.pdf
 Circuit Breaker: http://martinfowler.com/bliki/CircuitBreaker.html
 Release It! by Michael T. Nygard
 Netflix Tech Blog: http://techblog.netflix.com/
Questions?
syntaxerror@hu.ibm.com

More Related Content

What's hot

SaaS Enablement of your existing application (Cloud Slam 2010)
SaaS Enablement of your existing application (Cloud Slam 2010)SaaS Enablement of your existing application (Cloud Slam 2010)
SaaS Enablement of your existing application (Cloud Slam 2010)Nati Shalom
 
Best Practices for Cloud Managed Services Providers: The Path to CMP Success
Best Practices for Cloud Managed Services Providers: The Path to CMP SuccessBest Practices for Cloud Managed Services Providers: The Path to CMP Success
Best Practices for Cloud Managed Services Providers: The Path to CMP SuccessRightScale
 
From Grid to Cloud
From Grid to CloudFrom Grid to Cloud
From Grid to Cloudgojkoadzic
 
How Docker and ASP.NET Core will change the life of a Microsoft developer
How Docker and ASP.NET Core will change the life of a Microsoft developerHow Docker and ASP.NET Core will change the life of a Microsoft developer
How Docker and ASP.NET Core will change the life of a Microsoft developerAlex Thissen
 
Gali Reznik, Amdocs
Gali Reznik, Amdocs Gali Reznik, Amdocs
Gali Reznik, Amdocs RightScale
 
Operating Mission Critical Applications in the Face of Natural Disasters
Operating Mission Critical Applications in the Face of Natural DisastersOperating Mission Critical Applications in the Face of Natural Disasters
Operating Mission Critical Applications in the Face of Natural DisastersAmazon Web Services
 
Elastic APM: Combinalo con tus logs y métricas para una visibilidad completa
Elastic APM: Combinalo con tus logs y métricas para una visibilidad completaElastic APM: Combinalo con tus logs y métricas para una visibilidad completa
Elastic APM: Combinalo con tus logs y métricas para una visibilidad completaElasticsearch
 
Enterprise machine learning on k8s lessons learned and the road ahead
Enterprise machine learning on k8s   lessons learned and the road aheadEnterprise machine learning on k8s   lessons learned and the road ahead
Enterprise machine learning on k8s lessons learned and the road aheadTimothy Chen
 
Construire une plateforme d'observabilité centralisée
Construire une plateforme d'observabilité centraliséeConstruire une plateforme d'observabilité centralisée
Construire une plateforme d'observabilité centraliséeElasticsearch
 
Modern Cloud Fundamentals: Misconceptions and Industry Trends
Modern Cloud Fundamentals: Misconceptions and Industry TrendsModern Cloud Fundamentals: Misconceptions and Industry Trends
Modern Cloud Fundamentals: Misconceptions and Industry TrendsChristopher Bennage
 
Acc webinar deploying act! in a citrix environment
Acc webinar deploying act! in a citrix environmentAcc webinar deploying act! in a citrix environment
Acc webinar deploying act! in a citrix environmentJon Klubnik
 
Managing Multi-hypervisor OpenStack Cloud with Single Virtual Network
Managing Multi-hypervisor OpenStack Cloud with Single Virtual NetworkManaging Multi-hypervisor OpenStack Cloud with Single Virtual Network
Managing Multi-hypervisor OpenStack Cloud with Single Virtual NetworkPLUMgrid
 
El camino hacia la nube hibrida y abierta de Red Hat
El camino hacia la nube hibrida y abierta de Red HatEl camino hacia la nube hibrida y abierta de Red Hat
El camino hacia la nube hibrida y abierta de Red HatGeneXus
 
AWS e-Zest Cloud Event 2013 - AWS for Enterprises
AWS e-Zest Cloud Event 2013 - AWS for EnterprisesAWS e-Zest Cloud Event 2013 - AWS for Enterprises
AWS e-Zest Cloud Event 2013 - AWS for Enterprisese-Zest Solutions
 
Lap Around Windows Azure Business Edition Final
Lap Around Windows Azure Business Edition FinalLap Around Windows Azure Business Edition Final
Lap Around Windows Azure Business Edition FinalKyle Cressman
 
Keynote : évolution et vision d'Elastic Observability
Keynote : évolution et vision d'Elastic ObservabilityKeynote : évolution et vision d'Elastic Observability
Keynote : évolution et vision d'Elastic ObservabilityElasticsearch
 
CloudCrowd gigaSpaces Presentation
CloudCrowd gigaSpaces PresentationCloudCrowd gigaSpaces Presentation
CloudCrowd gigaSpaces PresentationNati Shalom
 

What's hot (20)

emc world keynote gelsinger
emc world keynote gelsinger emc world keynote gelsinger
emc world keynote gelsinger
 
SaaS Enablement of your existing application (Cloud Slam 2010)
SaaS Enablement of your existing application (Cloud Slam 2010)SaaS Enablement of your existing application (Cloud Slam 2010)
SaaS Enablement of your existing application (Cloud Slam 2010)
 
Best Practices for Cloud Managed Services Providers: The Path to CMP Success
Best Practices for Cloud Managed Services Providers: The Path to CMP SuccessBest Practices for Cloud Managed Services Providers: The Path to CMP Success
Best Practices for Cloud Managed Services Providers: The Path to CMP Success
 
From Grid to Cloud
From Grid to CloudFrom Grid to Cloud
From Grid to Cloud
 
How Docker and ASP.NET Core will change the life of a Microsoft developer
How Docker and ASP.NET Core will change the life of a Microsoft developerHow Docker and ASP.NET Core will change the life of a Microsoft developer
How Docker and ASP.NET Core will change the life of a Microsoft developer
 
Gali Reznik, Amdocs
Gali Reznik, Amdocs Gali Reznik, Amdocs
Gali Reznik, Amdocs
 
Operating Mission Critical Applications in the Face of Natural Disasters
Operating Mission Critical Applications in the Face of Natural DisastersOperating Mission Critical Applications in the Face of Natural Disasters
Operating Mission Critical Applications in the Face of Natural Disasters
 
Elastic APM: Combinalo con tus logs y métricas para una visibilidad completa
Elastic APM: Combinalo con tus logs y métricas para una visibilidad completaElastic APM: Combinalo con tus logs y métricas para una visibilidad completa
Elastic APM: Combinalo con tus logs y métricas para una visibilidad completa
 
Enterprise machine learning on k8s lessons learned and the road ahead
Enterprise machine learning on k8s   lessons learned and the road aheadEnterprise machine learning on k8s   lessons learned and the road ahead
Enterprise machine learning on k8s lessons learned and the road ahead
 
Construire une plateforme d'observabilité centralisée
Construire une plateforme d'observabilité centraliséeConstruire une plateforme d'observabilité centralisée
Construire une plateforme d'observabilité centralisée
 
Cloudforms Workshop
Cloudforms WorkshopCloudforms Workshop
Cloudforms Workshop
 
Cloud testing
Cloud testingCloud testing
Cloud testing
 
Modern Cloud Fundamentals: Misconceptions and Industry Trends
Modern Cloud Fundamentals: Misconceptions and Industry TrendsModern Cloud Fundamentals: Misconceptions and Industry Trends
Modern Cloud Fundamentals: Misconceptions and Industry Trends
 
Acc webinar deploying act! in a citrix environment
Acc webinar deploying act! in a citrix environmentAcc webinar deploying act! in a citrix environment
Acc webinar deploying act! in a citrix environment
 
Managing Multi-hypervisor OpenStack Cloud with Single Virtual Network
Managing Multi-hypervisor OpenStack Cloud with Single Virtual NetworkManaging Multi-hypervisor OpenStack Cloud with Single Virtual Network
Managing Multi-hypervisor OpenStack Cloud with Single Virtual Network
 
El camino hacia la nube hibrida y abierta de Red Hat
El camino hacia la nube hibrida y abierta de Red HatEl camino hacia la nube hibrida y abierta de Red Hat
El camino hacia la nube hibrida y abierta de Red Hat
 
AWS e-Zest Cloud Event 2013 - AWS for Enterprises
AWS e-Zest Cloud Event 2013 - AWS for EnterprisesAWS e-Zest Cloud Event 2013 - AWS for Enterprises
AWS e-Zest Cloud Event 2013 - AWS for Enterprises
 
Lap Around Windows Azure Business Edition Final
Lap Around Windows Azure Business Edition FinalLap Around Windows Azure Business Edition Final
Lap Around Windows Azure Business Edition Final
 
Keynote : évolution et vision d'Elastic Observability
Keynote : évolution et vision d'Elastic ObservabilityKeynote : évolution et vision d'Elastic Observability
Keynote : évolution et vision d'Elastic Observability
 
CloudCrowd gigaSpaces Presentation
CloudCrowd gigaSpaces PresentationCloudCrowd gigaSpaces Presentation
CloudCrowd gigaSpaces Presentation
 

Similar to Scalable service architectures @ VDB16

Scalable service architectures @ BWS16
Scalable service architectures @ BWS16Scalable service architectures @ BWS16
Scalable service architectures @ BWS16Zoltán Németh
 
Scalable Service Architectures
Scalable Service ArchitecturesScalable Service Architectures
Scalable Service ArchitecturesZoltán Németh
 
Microsoft Azure Cloud Basics Tutorial
Microsoft Azure Cloud Basics TutorialMicrosoft Azure Cloud Basics Tutorial
Microsoft Azure Cloud Basics TutorialIIMSE Edu
 
Stephane Lapointe & Alexandre Brisebois: Développer des microservices avec Se...
Stephane Lapointe & Alexandre Brisebois: Développer des microservices avec Se...Stephane Lapointe & Alexandre Brisebois: Développer des microservices avec Se...
Stephane Lapointe & Alexandre Brisebois: Développer des microservices avec Se...MSDEVMTL
 
Deploying SaaS Application on the Cloud - Case Study
Deploying SaaS Application on the Cloud - Case StudyDeploying SaaS Application on the Cloud - Case Study
Deploying SaaS Application on the Cloud - Case StudyNati Shalom
 
Deploy Microservices in the Real World
Deploy Microservices in the Real WorldDeploy Microservices in the Real World
Deploy Microservices in the Real WorldElana Krasner
 
Muves3 Elastic Grid Java One2009 Final
Muves3 Elastic Grid Java One2009 FinalMuves3 Elastic Grid Java One2009 Final
Muves3 Elastic Grid Java One2009 FinalElastic Grid, LLC.
 
Giga spaces value prop - afas - cloud practices
Giga spaces value prop - afas - cloud practicesGiga spaces value prop - afas - cloud practices
Giga spaces value prop - afas - cloud practicesTricode (part of Dept)
 
Giga Spaces Alternative To GAE_JavaOne 09
Giga Spaces Alternative To GAE_JavaOne 09Giga Spaces Alternative To GAE_JavaOne 09
Giga Spaces Alternative To GAE_JavaOne 09Amnon Raviv
 
Savig cost using application level virtualization
Savig cost using application level virtualizationSavig cost using application level virtualization
Savig cost using application level virtualizationNati Shalom
 
Why Cloud Management Makes Sense
Why Cloud Management Makes SenseWhy Cloud Management Makes Sense
Why Cloud Management Makes SenseRightScale
 
Karrox introduction to cloud computing
Karrox introduction to cloud computingKarrox introduction to cloud computing
Karrox introduction to cloud computingKarrox Franchise
 
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...Adrian Cockcroft
 
December 15 — Cloud Computing and Hosting
December 15 — Cloud Computing and HostingDecember 15 — Cloud Computing and Hosting
December 15 — Cloud Computing and Hostingwebhostingguy
 
Primatics Financial - Parallel, High Throughput Risk Calculations On The Cloud
Primatics Financial - Parallel, High Throughput Risk Calculations On The CloudPrimatics Financial - Parallel, High Throughput Risk Calculations On The Cloud
Primatics Financial - Parallel, High Throughput Risk Calculations On The CloudAmnon Raviv
 
GigaSpaces CCF 4 Xap
GigaSpaces CCF 4 XapGigaSpaces CCF 4 Xap
GigaSpaces CCF 4 XapShay Hassidim
 
Cloud Crowd GigaSpaces Presentation
Cloud Crowd GigaSpaces PresentationCloud Crowd GigaSpaces Presentation
Cloud Crowd GigaSpaces Presentationjimliddle
 

Similar to Scalable service architectures @ VDB16 (20)

Scalable service architectures @ BWS16
Scalable service architectures @ BWS16Scalable service architectures @ BWS16
Scalable service architectures @ BWS16
 
Scalable Service Architectures
Scalable Service ArchitecturesScalable Service Architectures
Scalable Service Architectures
 
Microsoft Azure Cloud Basics Tutorial
Microsoft Azure Cloud Basics TutorialMicrosoft Azure Cloud Basics Tutorial
Microsoft Azure Cloud Basics Tutorial
 
Stephane Lapointe & Alexandre Brisebois: Développer des microservices avec Se...
Stephane Lapointe & Alexandre Brisebois: Développer des microservices avec Se...Stephane Lapointe & Alexandre Brisebois: Développer des microservices avec Se...
Stephane Lapointe & Alexandre Brisebois: Développer des microservices avec Se...
 
Deploying SaaS Application on the Cloud - Case Study
Deploying SaaS Application on the Cloud - Case StudyDeploying SaaS Application on the Cloud - Case Study
Deploying SaaS Application on the Cloud - Case Study
 
Deploy Microservices in the Real World
Deploy Microservices in the Real WorldDeploy Microservices in the Real World
Deploy Microservices in the Real World
 
Muves3 Elastic Grid Java One2009 Final
Muves3 Elastic Grid Java One2009 FinalMuves3 Elastic Grid Java One2009 Final
Muves3 Elastic Grid Java One2009 Final
 
Giga spaces value prop - afas - cloud practices
Giga spaces value prop - afas - cloud practicesGiga spaces value prop - afas - cloud practices
Giga spaces value prop - afas - cloud practices
 
Cloud computing
Cloud computingCloud computing
Cloud computing
 
Introduction To Cloud Computing
Introduction To Cloud ComputingIntroduction To Cloud Computing
Introduction To Cloud Computing
 
Giga Spaces Alternative To GAE_JavaOne 09
Giga Spaces Alternative To GAE_JavaOne 09Giga Spaces Alternative To GAE_JavaOne 09
Giga Spaces Alternative To GAE_JavaOne 09
 
Savig cost using application level virtualization
Savig cost using application level virtualizationSavig cost using application level virtualization
Savig cost using application level virtualization
 
Why Cloud Management Makes Sense
Why Cloud Management Makes SenseWhy Cloud Management Makes Sense
Why Cloud Management Makes Sense
 
Karrox introduction to cloud computing
Karrox introduction to cloud computingKarrox introduction to cloud computing
Karrox introduction to cloud computing
 
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
CMG2013 Workshop: Netflix Cloud Native, Capacity, Performance and Cost Optimi...
 
Gogrid
GogridGogrid
Gogrid
 
December 15 — Cloud Computing and Hosting
December 15 — Cloud Computing and HostingDecember 15 — Cloud Computing and Hosting
December 15 — Cloud Computing and Hosting
 
Primatics Financial - Parallel, High Throughput Risk Calculations On The Cloud
Primatics Financial - Parallel, High Throughput Risk Calculations On The CloudPrimatics Financial - Parallel, High Throughput Risk Calculations On The Cloud
Primatics Financial - Parallel, High Throughput Risk Calculations On The Cloud
 
GigaSpaces CCF 4 Xap
GigaSpaces CCF 4 XapGigaSpaces CCF 4 Xap
GigaSpaces CCF 4 Xap
 
Cloud Crowd GigaSpaces Presentation
Cloud Crowd GigaSpaces PresentationCloud Crowd GigaSpaces Presentation
Cloud Crowd GigaSpaces Presentation
 

More from Zoltán Németh

Reveal The Secrets of Your Videos
Reveal The Secrets of Your VideosReveal The Secrets of Your Videos
Reveal The Secrets of Your VideosZoltán Németh
 
Voxxed Days Belgrade 2017 - How not to do DevOps
Voxxed Days Belgrade 2017 - How not to do DevOpsVoxxed Days Belgrade 2017 - How not to do DevOps
Voxxed Days Belgrade 2017 - How not to do DevOpsZoltán Németh
 
Content protection with UMS
Content protection with UMSContent protection with UMS
Content protection with UMSZoltán Németh
 
Implementing DevOps In Practice
Implementing DevOps In PracticeImplementing DevOps In Practice
Implementing DevOps In PracticeZoltán Németh
 
On-demand real time transcoding
On-demand real time transcoding On-demand real time transcoding
On-demand real time transcoding Zoltán Németh
 
DB séma kezelés Liquibase-el
DB séma kezelés Liquibase-elDB séma kezelés Liquibase-el
DB séma kezelés Liquibase-elZoltán Németh
 

More from Zoltán Németh (9)

Reveal The Secrets of Your Videos
Reveal The Secrets of Your VideosReveal The Secrets of Your Videos
Reveal The Secrets of Your Videos
 
Voxxed Days Belgrade 2017 - How not to do DevOps
Voxxed Days Belgrade 2017 - How not to do DevOpsVoxxed Days Belgrade 2017 - How not to do DevOps
Voxxed Days Belgrade 2017 - How not to do DevOps
 
Content protection with UMS
Content protection with UMSContent protection with UMS
Content protection with UMS
 
Implementing DevOps In Practice
Implementing DevOps In PracticeImplementing DevOps In Practice
Implementing DevOps In Practice
 
Building our own CDN
Building our own CDNBuilding our own CDN
Building our own CDN
 
Culture @ Velocity UK
Culture @ Velocity UKCulture @ Velocity UK
Culture @ Velocity UK
 
On-demand real time transcoding
On-demand real time transcoding On-demand real time transcoding
On-demand real time transcoding
 
DB séma kezelés Liquibase-el
DB séma kezelés Liquibase-elDB séma kezelés Liquibase-el
DB séma kezelés Liquibase-el
 
Daemons in PHP
Daemons in PHPDaemons in PHP
Daemons in PHP
 

Recently uploaded

DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Scott Andery
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dashnarutouzumaki53779
 

Recently uploaded (20)

DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dash
 

Scalable service architectures @ VDB16

  • 1. Scalable Service Architectures Lessons learned Zoltán Németh Sr Engineering Manager, Streaming & PlayerAn IBM company
  • 2.  Founded in 2007, acquired by IBM in 2016  Live streaming and VOD  Freemium / Pro / Demand / Align An IBM company
  • 4.
  • 5. Now back to 2009...
  • 6.
  • 7.
  • 8. Earthquake in Japan Protests in Ukraine, Egypt, Syria Asteroid Approach SpaceX Launch El Classico
  • 9.
  • 11. Defining scalability Scalability is the ability to handle increased workload by repeatedly applying a costeffective strategy for extending a system’s capacity. (CMU paper, 2006) How well a solution to some problem will work when the size of the problem increases. When the size decreases, the solution must fit. (dictionary.com and Theo Schlossnagle, 2006)
  • 12. Self-contained service  Explicitly declare and isolate dependencies  Isolation from the outside system  Static linking  Pay attention to GPL  Do not rely on system packages
  • 13. Disposability  Maximize robustness with fast startup and graceful shutdown  Disposable processes  Graceful shutdown on SIGTERM  Handling sudden death: robust queue backend
  • 14. Backing Services  Treat backing services as attached resources  No distinction between local and third party services  Easily swap out resources  Export services via port binding  Become the backing service for another app Drawing source: 12factor.net
  • 15. Processes, concurrency  Stateless processes (not even sticky sessions)  Process types by work type  We <3 linux process  Container > VM  Shared-nothing  adding concurrency is safe  Process distribution spanning machines
  • 16. Statelessness  Store everything in a datastore  Aggregate data  Aggregator / map & reduce  CQEngine  Chandra  Scalable datastores  Handling user sessions
  • 17. Microservices  Self-contained  Disposable  Stateless  Shared-nothing  API communication  Dependency management moved to external  Be Warned! Image credits: christofcoetzee.co.za, techblog.netflix.com
  • 18. Monitoring  Metrics collecting  Graphite, New Relic  Self-aware applications  Cluster state  Zookeeper, Consul  Scaling decisions  Capacity amount  Graph derivative  App requests
  • 19.
  • 20. Load Balance  DNS or API  App level balance  Uniform entry point or proxy  Balance decisions  Load  Zookeeper state  Resource policies
  • 21. Service Separation  Rate limiting  Failure is inevitable  Circuit Breaker pattern  Stop cascading failure, allow recovery  Fail fast, fail silent  Hystrix  Service decoupling  Asynchronous operations
  • 22. Deployment  Automate all the things  Chef & VMs  Docker  Immutable deployment  Docker / Kubernetes / Rancher  Handling tasks before shutdown
  • 23.
  • 24. Extras  Debugging features  Log processing: Logstash, Kibana  Clojure / JS consoles  Runtime configuration via env  Scaling API  Cloud providers  Automatic start / stop
  • 25. Reading  Scalable Internet Architectures by Theo Schlossnagle  The 12-factor App: http://12factor.net/  Carnegie Mellon Paper: http://www.sei.cmu.edu/reports/06tn012.pdf  Circuit Breaker: http://martinfowler.com/bliki/CircuitBreaker.html  Release It! by Michael T. Nygard  Netflix Tech Blog: http://techblog.netflix.com/

Editor's Notes

  1. A bit of Ustream intro
  2. Quick description of the streaming stack, roles of components, how they require scaling - Transcontroller/transcoder scaling - UMS scaling
  3. Quick description of the streaming stack, roles of components, how they require scaling - Transcontroller/transcoder scaling UMS scaling
  4. 30 day viewer graph. Clear peaks -> need for scaling
  5. Scaling delivery  CDN, UCDN, other talk  Scaling applications! Now comes some scaling theory
  6. Carnegie Mellon University paper by Charles B. Weinstock, John B. Goodenough: On System Scalability LINFO: The Linux Information Project http://www.linfo.org/ Next: principles
  7. Example: calling imagemagick or curl from code – they might be there or might not be Bundle everything into the app instead
  8. Disposable process: they can be started or stopped at a moment’s notice For a web process, graceful shutdown is achieved by ceasing to listen on the service port (thereby refusing any new requests), allowing any current requests to finish, and then exiting. Implicit in this model is that HTTP requests are short (no more than a few seconds), or in the case of long polling, the client should seamlessly attempt to reconnect when the connection is lost. For a worker process, graceful shutdown is achieved by returning the current job to the work queue.
  9. A backing service is any service the app consumes over the network as part of its normal operation. Examples include datastores (such as MySQL or CouchDB), messaging/queueing systems (such as RabbitMQ or Beanstalkd), SMTP services for outbound email (such as Postfix), and caching systems (such as Memcached). Put a resource locator in the config only – environment variables Example: Easily swap out a local mysql to a remote service The app does not rely on runtime injection of a webserver into the execution environment to create a web-facing service. The web app exports HTTP as a service by binding to a port, and listening to requests coming in on that port. One app can become the backing service for another app, by providing the URL to the backing app as a resource handle in the config for the consuming app
  10. Handle diverse workloads by assigning each type of work to a process type. For example, HTTP requests may be handled by a web process, and long-running background tasks handled by a worker process An individual VM can only grow so large (vertical scale), so the application must also be able to span multiple processes running on multiple physical machines.
  11. Aggregate everything within the app and write it out in bulk – careful about write frequency, must not lose too many data on a crash Aggregator map-reduce Redis: scales reads, write problematic Cassandra: quick scaling questionable Aerospike: scales reads and writes, working together with their eng team User sessions: persistent connection, NIO+
  12. Aggregate everything within the app and write it out in bulk – careful about write frequency, must not lose too many data on a crash Aggregator map-reduce Redis: scales reads, write problematic Cassandra: quick scaling questionable Aerospike: scales reads and writes, working together with their eng team User sessions: persistent connection, NIO+
  13. Report everything to graphite, constantly check graph trends automatically Apps are self-aware, they know their health App instances report into Zookeeper and thus know about each other Central logic can request resource based on capacity or graph, app can request based on self-check or zookeeper Zookeeper, Consul: miért, mik az előnyei
  14. load balancing distributes workloads across multiple computing resources Flexibility: can increase or decrease its own size, example: Threadpools Adapting to CPU, RAM, disk, network App level: transcontroller selects transcoder App level balance with proxy can be SPOF, careful Resource policies: even distribution, keep large chunks free for possible large tasks (transcoder use case), group requests together on some attribute (pro, etc)
  15. Failure inevitable because: large numbers, hw issues, independent network Hystrix by Netflix 2011/12 Circuit Breaker: Martin Fowler post from 2014 Decoupling: serving one request should not wait on others Service decoupling example: inserting layers between DB and UMS -> RGW. Then another layer between RGW and UMS -> Queue Antipattern example: connection limit, if filled up, new connections are kept waiting until a resource frees up
  16. Docker: build images from dockerfile, deploy from repository Tasks before shutdown: moving jobs, log collection, sleep
  17. van egy environment, es abba rakunk egy kubernetest, ez N darab gepen fut, abbol a nehany gepbol 1-3 az maga a Kubernetes master, a tobbi pedig a worker nodeok, azokon futnak a userek alkalmazasai. A kornyezetek jol elszeparaltak, nincsennek egymassal hatassal, de ha kell megtalaljak egymast A pod-ok azok kontenerek halmazai (1 vagy tobb, altalaban 1 amugy)[9:33] Peldaul az egy POD hogy UHS meg Chunkserver van egy gepen, mert a chunkserver olvassa az Ingest altal kiirt fajlokat.[9:33] Ez ket kontener, de megis egy POD. egy app az servicek es replication controllerek-bol allnak, a replication controller ami felugyeli hogy kello szamu instance legyen egy adott POD-bol (anya POD)
  18. Logs: logs as stream / stdout (factor #9), collect / transport / process Scaling API: Other considerations: price, network line to the cloud provider, instance type (spot vs normal) Openstack, Ganeti