SlideShare a Scribd company logo

Distributed Consensus A.K.A. "What do we eat for lunch?"

Distributed Consensus is everywhere! Even if not obvious at first, most apps nowadays are distributed systems, and these sometimes have to "agree on a value", this is where consensus algorithms come in. In this session we'll look at the general problem and solve a few example cases using the RAFT algorithm implemented using Akka's Actor and Cluster modules.

1 of 174
Download to read offline
Konrad 'ktoso' Malawski 
Distributed Consensus 
“What do we eat for lunch?” 
GeeCON 2014 @ Kraków, PL 
A.K.A. 
Konrad `@ktosopl` Malawski
Konrad 'ktoso' Malawski 
Distributed Consensus 
GeeCON 2014 @ Kraków, PL 
A.K.A. 
“What do we eat for lunch?” 
real world 
edition 
Konrad `@ktosopl` Malawski
hAkker @ 
Konrad `@ktosopl` Malawski
hAkker @ 
Konrad `@ktosopl` Malawski 
typesafe.com 
geecon.org 
Java.pl / KrakowScala.pl 
sckrk.com / meetup.com/Paper-Cup @ London 
GDGKrakow.pl 
meetup.com/Lambda-Lounge-Krakow
You? 
Distributed systems?
You? 
Distributed systems? 
?

Recommended

Need for Async: Hot pursuit for scalable applications
Need for Async: Hot pursuit for scalable applicationsNeed for Async: Hot pursuit for scalable applications
Need for Async: Hot pursuit for scalable applicationsKonrad Malawski
 
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka StreamsFresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka StreamsKonrad Malawski
 
The Need for Async @ ScalaWorld
The Need for Async @ ScalaWorldThe Need for Async @ ScalaWorld
The Need for Async @ ScalaWorldKonrad Malawski
 
ScalaSwarm 2017 Keynote: Tough this be madness yet theres method in't
ScalaSwarm 2017 Keynote: Tough this be madness yet theres method in'tScalaSwarm 2017 Keynote: Tough this be madness yet theres method in't
ScalaSwarm 2017 Keynote: Tough this be madness yet theres method in'tKonrad Malawski
 
Networks and Types - the Future of Akka @ ScalaDays NYC 2018
Networks and Types - the Future of Akka @ ScalaDays NYC 2018Networks and Types - the Future of Akka @ ScalaDays NYC 2018
Networks and Types - the Future of Akka @ ScalaDays NYC 2018Konrad Malawski
 
Building reactive distributed systems with Akka
Building reactive distributed systems with Akka Building reactive distributed systems with Akka
Building reactive distributed systems with Akka Johan Andrén
 
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYCBuilding a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYCKonrad Malawski
 
State of Akka 2017 - The best is yet to come
State of Akka 2017 - The best is yet to comeState of Akka 2017 - The best is yet to come
State of Akka 2017 - The best is yet to comeKonrad Malawski
 

More Related Content

What's hot

The things we don't see – stories of Software, Scala and Akka
The things we don't see – stories of Software, Scala and AkkaThe things we don't see – stories of Software, Scala and Akka
The things we don't see – stories of Software, Scala and AkkaKonrad Malawski
 
Not Only Streams for Akademia JLabs
Not Only Streams for Akademia JLabsNot Only Streams for Akademia JLabs
Not Only Streams for Akademia JLabsKonrad Malawski
 
Async - react, don't wait - PingConf
Async - react, don't wait - PingConfAsync - react, don't wait - PingConf
Async - react, don't wait - PingConfJohan Andrén
 
Building Reactive Systems with Akka (in Java 8 or Scala)
Building Reactive Systems with Akka (in Java 8 or Scala)Building Reactive Systems with Akka (in Java 8 or Scala)
Building Reactive Systems with Akka (in Java 8 or Scala)Jonas Bonér
 
The Cloud-natives are RESTless @ JavaOne
The Cloud-natives are RESTless @ JavaOneThe Cloud-natives are RESTless @ JavaOne
The Cloud-natives are RESTless @ JavaOneKonrad Malawski
 
The Newest in Session Types
The Newest in Session TypesThe Newest in Session Types
The Newest in Session TypesRoland Kuhn
 
The dark side of Akka and the remedy
The dark side of Akka and the remedyThe dark side of Akka and the remedy
The dark side of Akka and the remedykrivachy
 
Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...
Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...
Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...PROIDEA
 
2014 akka-streams-tokyo-japanese
2014 akka-streams-tokyo-japanese2014 akka-streams-tokyo-japanese
2014 akka-streams-tokyo-japaneseKonrad Malawski
 
Next generation actors with Akka
Next generation actors with AkkaNext generation actors with Akka
Next generation actors with AkkaJohan Andrén
 
Networks and types - the future of Akka
Networks and types - the future of AkkaNetworks and types - the future of Akka
Networks and types - the future of AkkaJohan Andrén
 
Streaming all the things with akka streams
Streaming all the things with akka streams   Streaming all the things with akka streams
Streaming all the things with akka streams Johan Andrén
 
Introduction to Akka - Atlanta Java Users Group
Introduction to Akka - Atlanta Java Users GroupIntroduction to Akka - Atlanta Java Users Group
Introduction to Akka - Atlanta Java Users GroupRoy Russo
 
Real world functional reactive programming
Real world functional reactive programmingReal world functional reactive programming
Real world functional reactive programmingEric Polerecky
 
Async – react, don't wait
Async – react, don't waitAsync – react, don't wait
Async – react, don't waitJohan Andrén
 
Next generation message driven systems with Akka
Next generation message driven systems with AkkaNext generation message driven systems with Akka
Next generation message driven systems with AkkaJohan Andrén
 
Reactive Programming with Rx
 Reactive Programming with Rx Reactive Programming with Rx
Reactive Programming with RxC4Media
 
Akka Futures and Akka Remoting
Akka Futures  and Akka RemotingAkka Futures  and Akka Remoting
Akka Futures and Akka RemotingKnoldus Inc.
 
Samuele Resca - REACTIVE PROGRAMMING, DAMN. IT IS NOT ABOUT REACTJS - Codemot...
Samuele Resca - REACTIVE PROGRAMMING, DAMN. IT IS NOT ABOUT REACTJS - Codemot...Samuele Resca - REACTIVE PROGRAMMING, DAMN. IT IS NOT ABOUT REACTJS - Codemot...
Samuele Resca - REACTIVE PROGRAMMING, DAMN. IT IS NOT ABOUT REACTJS - Codemot...Codemotion
 

What's hot (20)

The things we don't see – stories of Software, Scala and Akka
The things we don't see – stories of Software, Scala and AkkaThe things we don't see – stories of Software, Scala and Akka
The things we don't see – stories of Software, Scala and Akka
 
Not Only Streams for Akademia JLabs
Not Only Streams for Akademia JLabsNot Only Streams for Akademia JLabs
Not Only Streams for Akademia JLabs
 
Async - react, don't wait - PingConf
Async - react, don't wait - PingConfAsync - react, don't wait - PingConf
Async - react, don't wait - PingConf
 
Building Reactive Systems with Akka (in Java 8 or Scala)
Building Reactive Systems with Akka (in Java 8 or Scala)Building Reactive Systems with Akka (in Java 8 or Scala)
Building Reactive Systems with Akka (in Java 8 or Scala)
 
The Cloud-natives are RESTless @ JavaOne
The Cloud-natives are RESTless @ JavaOneThe Cloud-natives are RESTless @ JavaOne
The Cloud-natives are RESTless @ JavaOne
 
Actor Model Akka Framework
Actor Model Akka FrameworkActor Model Akka Framework
Actor Model Akka Framework
 
The Newest in Session Types
The Newest in Session TypesThe Newest in Session Types
The Newest in Session Types
 
The dark side of Akka and the remedy
The dark side of Akka and the remedyThe dark side of Akka and the remedy
The dark side of Akka and the remedy
 
Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...
Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...
Atmosphere Conference 2015: Need for Async: In pursuit of scalable internet-s...
 
2014 akka-streams-tokyo-japanese
2014 akka-streams-tokyo-japanese2014 akka-streams-tokyo-japanese
2014 akka-streams-tokyo-japanese
 
Next generation actors with Akka
Next generation actors with AkkaNext generation actors with Akka
Next generation actors with Akka
 
Networks and types - the future of Akka
Networks and types - the future of AkkaNetworks and types - the future of Akka
Networks and types - the future of Akka
 
Streaming all the things with akka streams
Streaming all the things with akka streams   Streaming all the things with akka streams
Streaming all the things with akka streams
 
Introduction to Akka - Atlanta Java Users Group
Introduction to Akka - Atlanta Java Users GroupIntroduction to Akka - Atlanta Java Users Group
Introduction to Akka - Atlanta Java Users Group
 
Real world functional reactive programming
Real world functional reactive programmingReal world functional reactive programming
Real world functional reactive programming
 
Async – react, don't wait
Async – react, don't waitAsync – react, don't wait
Async – react, don't wait
 
Next generation message driven systems with Akka
Next generation message driven systems with AkkaNext generation message driven systems with Akka
Next generation message driven systems with Akka
 
Reactive Programming with Rx
 Reactive Programming with Rx Reactive Programming with Rx
Reactive Programming with Rx
 
Akka Futures and Akka Remoting
Akka Futures  and Akka RemotingAkka Futures  and Akka Remoting
Akka Futures and Akka Remoting
 
Samuele Resca - REACTIVE PROGRAMMING, DAMN. IT IS NOT ABOUT REACTJS - Codemot...
Samuele Resca - REACTIVE PROGRAMMING, DAMN. IT IS NOT ABOUT REACTJS - Codemot...Samuele Resca - REACTIVE PROGRAMMING, DAMN. IT IS NOT ABOUT REACTJS - Codemot...
Samuele Resca - REACTIVE PROGRAMMING, DAMN. IT IS NOT ABOUT REACTJS - Codemot...
 

Viewers also liked

From Microliths To Microsystems
From Microliths To MicrosystemsFrom Microliths To Microsystems
From Microliths To MicrosystemsJonas Bonér
 
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)Konrad Malawski
 
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRKKonrad Malawski
 
[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...
[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...
[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...Konrad Malawski
 
Akka Streams in Action @ ScalaDays Berlin 2016
Akka Streams in Action @ ScalaDays Berlin 2016Akka Streams in Action @ ScalaDays Berlin 2016
Akka Streams in Action @ ScalaDays Berlin 2016Konrad Malawski
 
How Reactive Streams & Akka Streams change the JVM Ecosystem
How Reactive Streams & Akka Streams change the JVM EcosystemHow Reactive Streams & Akka Streams change the JVM Ecosystem
How Reactive Streams & Akka Streams change the JVM EcosystemKonrad Malawski
 
DDDing Tools = Akka Persistence
DDDing Tools = Akka PersistenceDDDing Tools = Akka Persistence
DDDing Tools = Akka PersistenceKonrad Malawski
 
End to End Akka Streams / Reactive Streams - from Business to Socket
End to End Akka Streams / Reactive Streams - from Business to SocketEnd to End Akka Streams / Reactive Streams - from Business to Socket
End to End Akka Streams / Reactive Streams - from Business to SocketKonrad Malawski
 
Reactive Streams, j.u.concurrent & Beyond!
Reactive Streams, j.u.concurrent & Beyond!Reactive Streams, j.u.concurrent & Beyond!
Reactive Streams, j.u.concurrent & Beyond!Konrad Malawski
 
Reactive Streams / Akka Streams - GeeCON Prague 2014
Reactive Streams / Akka Streams - GeeCON Prague 2014Reactive Streams / Akka Streams - GeeCON Prague 2014
Reactive Streams / Akka Streams - GeeCON Prague 2014Konrad Malawski
 
Reactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka StreamsReactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka StreamsKonrad Malawski
 
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems
Go Reactive: Event-Driven, Scalable, Resilient & Responsive SystemsGo Reactive: Event-Driven, Scalable, Resilient & Responsive Systems
Go Reactive: Event-Driven, Scalable, Resilient & Responsive SystemsJonas Bonér
 
Akka persistence == event sourcing in 30 minutes
Akka persistence == event sourcing in 30 minutesAkka persistence == event sourcing in 30 minutes
Akka persistence == event sourcing in 30 minutesKonrad Malawski
 
Reactive integrations with Akka Streams
Reactive integrations with Akka StreamsReactive integrations with Akka Streams
Reactive integrations with Akka StreamsKonrad Malawski
 
Without Resilience, Nothing Else Matters
Without Resilience, Nothing Else MattersWithout Resilience, Nothing Else Matters
Without Resilience, Nothing Else MattersJonas Bonér
 
Life Beyond the Illusion of Present
Life Beyond the Illusion of PresentLife Beyond the Illusion of Present
Life Beyond the Illusion of PresentJonas Bonér
 
Akka-chan's Survival Guide for the Streaming World
Akka-chan's Survival Guide for the Streaming WorldAkka-chan's Survival Guide for the Streaming World
Akka-chan's Survival Guide for the Streaming WorldKonrad Malawski
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsJonas Bonér
 
STM in Haskell
STM in HaskellSTM in Haskell
STM in Haskellbegriffs
 
Krakow communities @ 2016
Krakow communities @ 2016Krakow communities @ 2016
Krakow communities @ 2016Konrad Malawski
 

Viewers also liked (20)

From Microliths To Microsystems
From Microliths To MicrosystemsFrom Microliths To Microsystems
From Microliths To Microsystems
 
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
 
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK
 
[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...
[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...
[Japanese] How Reactive Streams and Akka Streams change the JVM Ecosystem @ R...
 
Akka Streams in Action @ ScalaDays Berlin 2016
Akka Streams in Action @ ScalaDays Berlin 2016Akka Streams in Action @ ScalaDays Berlin 2016
Akka Streams in Action @ ScalaDays Berlin 2016
 
How Reactive Streams & Akka Streams change the JVM Ecosystem
How Reactive Streams & Akka Streams change the JVM EcosystemHow Reactive Streams & Akka Streams change the JVM Ecosystem
How Reactive Streams & Akka Streams change the JVM Ecosystem
 
DDDing Tools = Akka Persistence
DDDing Tools = Akka PersistenceDDDing Tools = Akka Persistence
DDDing Tools = Akka Persistence
 
End to End Akka Streams / Reactive Streams - from Business to Socket
End to End Akka Streams / Reactive Streams - from Business to SocketEnd to End Akka Streams / Reactive Streams - from Business to Socket
End to End Akka Streams / Reactive Streams - from Business to Socket
 
Reactive Streams, j.u.concurrent & Beyond!
Reactive Streams, j.u.concurrent & Beyond!Reactive Streams, j.u.concurrent & Beyond!
Reactive Streams, j.u.concurrent & Beyond!
 
Reactive Streams / Akka Streams - GeeCON Prague 2014
Reactive Streams / Akka Streams - GeeCON Prague 2014Reactive Streams / Akka Streams - GeeCON Prague 2014
Reactive Streams / Akka Streams - GeeCON Prague 2014
 
Reactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka StreamsReactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka Streams
 
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems
Go Reactive: Event-Driven, Scalable, Resilient & Responsive SystemsGo Reactive: Event-Driven, Scalable, Resilient & Responsive Systems
Go Reactive: Event-Driven, Scalable, Resilient & Responsive Systems
 
Akka persistence == event sourcing in 30 minutes
Akka persistence == event sourcing in 30 minutesAkka persistence == event sourcing in 30 minutes
Akka persistence == event sourcing in 30 minutes
 
Reactive integrations with Akka Streams
Reactive integrations with Akka StreamsReactive integrations with Akka Streams
Reactive integrations with Akka Streams
 
Without Resilience, Nothing Else Matters
Without Resilience, Nothing Else MattersWithout Resilience, Nothing Else Matters
Without Resilience, Nothing Else Matters
 
Life Beyond the Illusion of Present
Life Beyond the Illusion of PresentLife Beyond the Illusion of Present
Life Beyond the Illusion of Present
 
Akka-chan's Survival Guide for the Streaming World
Akka-chan's Survival Guide for the Streaming WorldAkka-chan's Survival Guide for the Streaming World
Akka-chan's Survival Guide for the Streaming World
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 
STM in Haskell
STM in HaskellSTM in Haskell
STM in Haskell
 
Krakow communities @ 2016
Krakow communities @ 2016Krakow communities @ 2016
Krakow communities @ 2016
 

Similar to Distributed Consensus A.K.A. "What do we eat for lunch?"

Hadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm ArchitectureHadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm ArchitectureP. Taylor Goetz
 
Legacy projects: how to win the race
Legacy projects: how to win the raceLegacy projects: how to win the race
Legacy projects: how to win the raceVictor_Cr
 
Tour of language landscape (katsconf)
Tour of language landscape (katsconf)Tour of language landscape (katsconf)
Tour of language landscape (katsconf)Yan Cui
 
Threat stack aws
Threat stack awsThreat stack aws
Threat stack awsJen Andre
 
F# at GameSys
F# at GameSysF# at GameSys
F# at GameSysYan Cui
 
The hangover: A "modern" (?) high performance approach to build an offensive ...
The hangover: A "modern" (?) high performance approach to build an offensive ...The hangover: A "modern" (?) high performance approach to build an offensive ...
The hangover: A "modern" (?) high performance approach to build an offensive ...Nelson Brito
 
Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6While42
 
Idioms in swift 2016 05c
Idioms in swift 2016 05cIdioms in swift 2016 05c
Idioms in swift 2016 05cKaz Yoshikawa
 
Beware: Sharp Tools
Beware: Sharp ToolsBeware: Sharp Tools
Beware: Sharp Toolschrismdp
 
Esoteric, Obfuscated, Artistic Programming in Ruby
Esoteric, Obfuscated, Artistic Programming in RubyEsoteric, Obfuscated, Artistic Programming in Ruby
Esoteric, Obfuscated, Artistic Programming in Rubymametter
 
Let'swift "Concurrency in swift"
Let'swift "Concurrency in swift"Let'swift "Concurrency in swift"
Let'swift "Concurrency in swift"Hyuk Hur
 
A gentle introduction into AKKA and the actor model
A gentle introduction into AKKA and the actor modelA gentle introduction into AKKA and the actor model
A gentle introduction into AKKA and the actor modelMykhailo Kotsur
 
Logic programming a ruby perspective
Logic programming a ruby perspectiveLogic programming a ruby perspective
Logic programming a ruby perspectiveNorman Richards
 
Computer network (8)
Computer network (8)Computer network (8)
Computer network (8)NYversity
 
A Taste of Python - Devdays Toronto 2009
A Taste of Python - Devdays Toronto 2009A Taste of Python - Devdays Toronto 2009
A Taste of Python - Devdays Toronto 2009Jordan Baker
 
A Scala Corrections Library
A Scala Corrections LibraryA Scala Corrections Library
A Scala Corrections LibraryPaul Phillips
 
JVMLS 2016. Coroutines in Kotlin
JVMLS 2016. Coroutines in KotlinJVMLS 2016. Coroutines in Kotlin
JVMLS 2016. Coroutines in KotlinAndrey Breslav
 
What we Learned Implementing Puppet at Backstop
What we Learned Implementing Puppet at BackstopWhat we Learned Implementing Puppet at Backstop
What we Learned Implementing Puppet at BackstopPuppet
 

Similar to Distributed Consensus A.K.A. "What do we eat for lunch?" (20)

Hadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm ArchitectureHadoop Summit Europe 2014: Apache Storm Architecture
Hadoop Summit Europe 2014: Apache Storm Architecture
 
Legacy projects: how to win the race
Legacy projects: how to win the raceLegacy projects: how to win the race
Legacy projects: how to win the race
 
Tour of language landscape (katsconf)
Tour of language landscape (katsconf)Tour of language landscape (katsconf)
Tour of language landscape (katsconf)
 
Threat stack aws
Threat stack awsThreat stack aws
Threat stack aws
 
F# at GameSys
F# at GameSysF# at GameSys
F# at GameSys
 
The hangover: A "modern" (?) high performance approach to build an offensive ...
The hangover: A "modern" (?) high performance approach to build an offensive ...The hangover: A "modern" (?) high performance approach to build an offensive ...
The hangover: A "modern" (?) high performance approach to build an offensive ...
 
Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6
 
Idioms in swift 2016 05c
Idioms in swift 2016 05cIdioms in swift 2016 05c
Idioms in swift 2016 05c
 
Beware: Sharp Tools
Beware: Sharp ToolsBeware: Sharp Tools
Beware: Sharp Tools
 
Esoteric, Obfuscated, Artistic Programming in Ruby
Esoteric, Obfuscated, Artistic Programming in RubyEsoteric, Obfuscated, Artistic Programming in Ruby
Esoteric, Obfuscated, Artistic Programming in Ruby
 
Let'swift "Concurrency in swift"
Let'swift "Concurrency in swift"Let'swift "Concurrency in swift"
Let'swift "Concurrency in swift"
 
A gentle introduction into AKKA and the actor model
A gentle introduction into AKKA and the actor modelA gentle introduction into AKKA and the actor model
A gentle introduction into AKKA and the actor model
 
Profiling for Grown-Ups
Profiling for Grown-UpsProfiling for Grown-Ups
Profiling for Grown-Ups
 
Logic programming a ruby perspective
Logic programming a ruby perspectiveLogic programming a ruby perspective
Logic programming a ruby perspective
 
Computer network (8)
Computer network (8)Computer network (8)
Computer network (8)
 
A Taste of Python - Devdays Toronto 2009
A Taste of Python - Devdays Toronto 2009A Taste of Python - Devdays Toronto 2009
A Taste of Python - Devdays Toronto 2009
 
A Scala Corrections Library
A Scala Corrections LibraryA Scala Corrections Library
A Scala Corrections Library
 
Self healing data
Self healing dataSelf healing data
Self healing data
 
JVMLS 2016. Coroutines in Kotlin
JVMLS 2016. Coroutines in KotlinJVMLS 2016. Coroutines in Kotlin
JVMLS 2016. Coroutines in Kotlin
 
What we Learned Implementing Puppet at Backstop
What we Learned Implementing Puppet at BackstopWhat we Learned Implementing Puppet at Backstop
What we Learned Implementing Puppet at Backstop
 

Recently uploaded

Confoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data scienceConfoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data scienceSusan Ibach
 
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlueCloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlueShapeBlue
 
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...DianaGray10
 
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueCloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueShapeBlue
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriSafe Software
 
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...Neo4j
 
Roundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdfRoundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdfMostafa Higazy
 
iOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostingeriOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostingerssuser9354ce
 
AI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientAI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientKari Kakkonen
 
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...2toLead Limited
 
National Institute of Standards and Technology (NIST) Cybersecurity Framework...
National Institute of Standards and Technology (NIST) Cybersecurity Framework...National Institute of Standards and Technology (NIST) Cybersecurity Framework...
National Institute of Standards and Technology (NIST) Cybersecurity Framework...MichaelBenis1
 
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Product School
 
Transcript: Trending now: Book subjects on the move in the Canadian market - ...
Transcript: Trending now: Book subjects on the move in the Canadian market - ...Transcript: Trending now: Book subjects on the move in the Canadian market - ...
Transcript: Trending now: Book subjects on the move in the Canadian market - ...BookNet Canada
 
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024BookNet Canada
 
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...htrindia
 
What’s New in CloudStack 4.19, Abhishek Kumar, Release Manager Apache CloudSt...
What’s New in CloudStack 4.19, Abhishek Kumar, Release Manager Apache CloudSt...What’s New in CloudStack 4.19, Abhishek Kumar, Release Manager Apache CloudSt...
What’s New in CloudStack 4.19, Abhishek Kumar, Release Manager Apache CloudSt...ShapeBlue
 
New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024ThousandEyes
 
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...ShapeBlue
 
IT Nation Evolve event 2024 - Quarter 1
IT Nation Evolve event 2024  - Quarter 1IT Nation Evolve event 2024  - Quarter 1
IT Nation Evolve event 2024 - Quarter 1Inbay UK
 
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxGraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxNeo4j
 

Recently uploaded (20)

Confoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data scienceConfoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data science
 
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlueCloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
 
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
 
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueCloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
 
Roundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdfRoundtable_-_API_Research__Testing_Tools.pdf
Roundtable_-_API_Research__Testing_Tools.pdf
 
iOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostingeriOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostinger
 
AI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientAI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficient
 
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
 
National Institute of Standards and Technology (NIST) Cybersecurity Framework...
National Institute of Standards and Technology (NIST) Cybersecurity Framework...National Institute of Standards and Technology (NIST) Cybersecurity Framework...
National Institute of Standards and Technology (NIST) Cybersecurity Framework...
 
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
 
Transcript: Trending now: Book subjects on the move in the Canadian market - ...
Transcript: Trending now: Book subjects on the move in the Canadian market - ...Transcript: Trending now: Book subjects on the move in the Canadian market - ...
Transcript: Trending now: Book subjects on the move in the Canadian market - ...
 
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024
 
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
 
What’s New in CloudStack 4.19, Abhishek Kumar, Release Manager Apache CloudSt...
What’s New in CloudStack 4.19, Abhishek Kumar, Release Manager Apache CloudSt...What’s New in CloudStack 4.19, Abhishek Kumar, Release Manager Apache CloudSt...
What’s New in CloudStack 4.19, Abhishek Kumar, Release Manager Apache CloudSt...
 
New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024New ThousandEyes Product Features and Release Highlights: February 2024
New ThousandEyes Product Features and Release Highlights: February 2024
 
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
 
IT Nation Evolve event 2024 - Quarter 1
IT Nation Evolve event 2024  - Quarter 1IT Nation Evolve event 2024  - Quarter 1
IT Nation Evolve event 2024 - Quarter 1
 
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxGraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
 

Distributed Consensus A.K.A. "What do we eat for lunch?"

  • 1. Konrad 'ktoso' Malawski Distributed Consensus “What do we eat for lunch?” GeeCON 2014 @ Kraków, PL A.K.A. Konrad `@ktosopl` Malawski
  • 2. Konrad 'ktoso' Malawski Distributed Consensus GeeCON 2014 @ Kraków, PL A.K.A. “What do we eat for lunch?” real world edition Konrad `@ktosopl` Malawski
  • 3. hAkker @ Konrad `@ktosopl` Malawski
  • 4. hAkker @ Konrad `@ktosopl` Malawski typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com / meetup.com/Paper-Cup @ London GDGKrakow.pl meetup.com/Lambda-Lounge-Krakow
  • 8. What is this talk about? The network. ! How to think about distributed systems. ! Some healthy madness. Code in slides covers only “simplest possible case”.
  • 9. Ordering[T] Slightly chronological. ! By no means is it “worst to best”.
  • 11. Consensus - informal “we all agree on something”
  • 12. Consensus - formal Termination Every correct process decides some value. ! Validity If all correct processes propose the same value v, then all correct processes decide v. ! Integrity If a correct process decides v, then v must have been proposed by some correct process. ! Agreement Every correct process must agree on the same value.
  • 16. Distributed Consensus What is a distributed system anyway?
  • 17. Distributed system definition A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable. — Leslie Lamport http://research.microsoft.com/en-us/um/people/lamport/pubs/distributed-system.txt
  • 18. Distributed system definition A system in which participants communicate asynchronously using messages. http://research.microsoft.com/en-us/um/people/lamport/pubs/distributed-system.txt
  • 19. Distributed Systems - failure detection
  • 20. Distributed Systems - failure detection
  • 21. Distributed Systems - failure detection Jim had quit CorpSoft a while ago, but no-one ever told Bob…
  • 22. Distributed Systems - failure detection
  • 23. Distributed Systems - failure detection Failure detection: • can only rely on external knowledge • but what if there’s no-one to tell you? • thus: must be in-some-way time based
  • 25. Two Generals Problem Yellow and Blue armies must attack Pink City. They must attack together, otherwise they’ll die in vain. Now they must agree on the exact time of the attack. ! They can only send messengers, which Pink may intercept and kill.
  • 27. Two Generals Problem - happy case I need to inform blue about my attack plan. I don’t know when yellow will attack…
  • 28. Two Generals Problem - happy case
  • 29. 1) Initial message not lost
  • 30. Two Generals Problem - happy case I don’t know if Blue will also attack at 13:37… I’ll wait until I hear back from him.
  • 31. Two Generals Problem - happy case I don’t know if Blue will also attack at 13:37… I’ll wait until I hear back from him. Why?
  • 32. 2) Message might have not reached blue
  • 33. Blue must confirm the reception of the command
  • 34. 1) Yellow is now sure, but Blue isn’t!
  • 35. 1) Yellow is now sure, but Blue isn’t! Why?
  • 36. 2) Blue’s confirmation might have been lost!
  • 37. This is exactly mirrors the initial situation!
  • 38. 2 Generals Problem Translated to Akka
  • 39. 2 Generals translated to Akka: Akka Actors implement the Actor Model: ! Actors: • communicate via messages • create other actors • change their behaviour on receiving a msg !
  • 40. 2 Generals translated to Akka: Akka Actors implement the Actor Model: ! Actors: • communicate via messages • create other actors • change their behaviour on receiving a msg ! Gains? Distribution / separation / modelling abstraction
  • 41. 2 Generals translated to Akka: case class AttackAt(when: Date) Presentation–sized–snippet = does not cover all cases
  • 42. 2 Generals translated to Akka: ! ! class General(general: Option[ActorRef]) extends Actor {! ! ! val WhenIWantToAttack: Date = ???! ! general foreach { _ ! AttackAt(WhenIWantToAttack) }! ! def receive = {! case AttackAt(when) =>! println(s”General ${otherGeneralName} attacks at $when”)! ! ! ! println(s”I must confirm this!")! ! sender() ! AttackAt(when)! }! ! def otherGeneralName = ! ! ! ! if(self.path.name == “blue")!“yellow" else "blue"! }! Presentation–sized–snippet = does not cover all cases
  • 43. 2 Generals translated to Akka: ! ! class General(general: Option[ActorRef]) extends Actor {! ! ! val WhenIWantToAttack: Date = ???! ! general foreach { _ ! AttackAt(WhenIWantToAttack) }! ! def receive = {! case AttackAt(when) =>! println(s”General ${otherGeneralName} attacks at $when”)! ! ! ! println(s”I must confirm this!")! ! sender() ! AttackAt(when)! }! ! def otherGeneralName = ! ! ! ! if(self.path.name == “blue")!“yellow" else "blue"! }! Presentation–sized–snippet = does not cover all cases
  • 44. 2 Generals translated to Akka: ! ! class General(general: Option[ActorRef]) extends Actor {! ! ! val WhenIWantToAttack: Date = ???! ! general foreach { _ ! AttackAt(WhenIWantToAttack) }! ! def receive = {! case AttackAt(when) =>! println(s”General ${otherGeneralName} attacks at $when”)! ! ! ! println(s”I must confirm this!")! ! sender() ! AttackAt(when)! }! ! def otherGeneralName = ! ! ! ! if(self.path.name == “blue")!“yellow" else "blue"! }! Presentation–sized–snippet = does not cover all cases
  • 45. 2 Generals translated to Akka: ! ! class General(general: Option[ActorRef]) extends Actor {! ! ! val WhenIWantToAttack: Date = ???! ! general foreach { _ ! AttackAt(WhenIWantToAttack) }! ! def receive = {! case AttackAt(when) =>! println(s”General ${otherGeneralName} attacks at $when”)! ! ! ! println(s”I must confirm this!")! ! sender() ! AttackAt(when)! }! ! def otherGeneralName = ! ! ! ! if (self.path.name == “blue")!"yellow" else "blue"! }! Presentation–sized–snippet = does not cover all cases
  • 46. 2 Generals translated to Akka: val system = ActorSystem("two-generals")! ! val blue = ! system.actorOf(Props(new General(general = None)), name = "blue")! ! val yellow = ! system.actorOf(Props(new General(Some(blue))), name = "yellow")! The blue general attacks at 13:37, I must confirm this!! The yellow general attacks at 13:37, I must confirm this!! The blue general attacks at 13:37, I must confirm this!! ... Presentation–sized–snippet = does not cover all cases
  • 47. 8 Fallacies of Distributed Computing
  • 48. 8 Fallacies of Distributed Computing 1. The network is reliable. 2. Latency is zero. 3. Bandwidth is infinite. 4. The network is secure. 5. Topology doesn’t change. 6. There is one administrator. 7. Transport cost is zero. 8. The network is homogeneous. Peter Deutsch “The Eight Fallacies of Distributed Computing” https://blogs.oracle.com/jag/resource/Fallacies.html
  • 50. Failure models: Fail – Stop Fail – Recover Byzantine
  • 51. Failure models: Fail – Stop Fail – Recover Byzantine
  • 52. Failure models: Fail – Stop Fail – Recover Byzantine
  • 53. Failure models: Fail – Stop Fail – Recover Byzantine
  • 55. 2PC - step 1: Propose value
  • 56. 2PC - step 1: Propose value
  • 57. 2PC - step 1: Promise to agree on write
  • 58. 2PC - step 2: Commit the write
  • 59. 2PC - step 1: Propose value, and die
  • 60. 2PC - step 1: Propose value to 1 node, and die
  • 61. 2PC: Prepare needs timeouts
  • 62. 2PC: Timeouts + recovery committer
  • 63. 2PC: Timeouts + recovery committer
  • 64. 2PC: Timeouts + recovery committer
  • 65. 2PC: Timeouts + recovery committer
  • 66. 2PC: Timeouts + recovery committer
  • 67. Still can’t tolerate if the “accepted value” Actor dies
  • 68. 2PC: Timeouts + recovery committer
  • 69. 2PC: Timeouts + recovery committer
  • 70. 2 Phase Commit translated to Akka
  • 71. 2PC translated to Akka case class Prepare(value: Any)! case object Commit! ! sealed class AcceptorStatus! case object Prepared extends AcceptorStatus! case object Conflict extends AcceptorStatus! ! Presentation–sized–snippet = does not cover all cases
  • 72. 2PC translated to Akka case class Prepare(value: Any)! case object Commit! ! sealed class AcceptorStatus! case object Prepared extends AcceptorStatus! case object Conflict extends AcceptorStatus! ! Presentation–sized–snippet = does not cover all cases
  • 73. 2PC translated to Akka class Proposer(acceptors: List[ActorRef]) extends Actor {! var transactionId = 0! var preparedAcceptors = 0! ! def receive = {! case value: String =>! transactionId += 1! acceptors foreach { _ ! Prepare(transactionId, value) }! ! case Prepared =>! preparedAcceptors += 1! ! if (preparedAcceptors == acceptors.size)! acceptors foreach { _ ! Commit }! ! case Conflict =>! ! ! ! ! ! context stop self! }! }! Presentation–sized–snippet = does not cover all cases
  • 74. 2PC translated to Akka class Proposer(acceptors: List[ActorRef]) extends Actor {! var transactionId = 0! var preparedAcceptors = 0! ! def receive = {! case value: String =>! transactionId += 1! acceptors foreach { _ ! Prepare(transactionId, value) }! ! case Prepared =>! preparedAcceptors += 1! ! if (preparedAcceptors == acceptors.size)! acceptors foreach { _ ! Commit }! ! case Conflict =>! ! ! ! ! ! context stop self! }! }! Presentation–sized–snippet = does not cover all cases
  • 75. 2PC translated to Akka class Proposer(acceptors: List[ActorRef]) extends Actor {! var transactionId = 0! var preparedAcceptors = 0! ! def receive = {! case value: String =>! transactionId += 1! acceptors foreach { _ ! Prepare(transactionId, value) }! ! case Prepared =>! preparedAcceptors += 1! ! if (preparedAcceptors == acceptors.size)! acceptors foreach { _ ! Commit }! ! case Conflict =>! ! ! ! ! ! context stop self! }! }! Presentation–sized–snippet = does not cover all cases
  • 76. 2PC with ResumeProposer in Akka case class Prepare(value: Any)! case object Commit! ! sealed class AcceptorStatus! case object Prepared extends AcceptorStatus! case object Conflict extends AcceptorStatus! case class Committed(value: Any) extends AcceptorStatus! Presentation–sized–snippet = does not cover all cases
  • 77. 2PC with ResumeProposer in Akka ! class ResumeProposer(! proposer: ActorRef, ! acceptors: List[ActorRef]) extends Actor {! ! context watch proposer! ! var anyAcceptorCommitted = false! ! def receive = {! case Terminated(`proposer`) =>! println("Proposer died! Try to finish the transaction...")! acceptors map { _ ! StatusPlz }! ! case _: AcceptorStatus =>! // impl of recovery here! }! } Presentation–sized–snippet = does not cover all cases
  • 78. 2PC with ResumeProposer in Akka Presentation–sized–snippet = does not cover all cases
  • 80. Quorum voting From the perspective of the Omnipotent Observer *
  • 81. Quorum voting From the perspective of the Omnipotent Observer * * does not exist in a running system
  • 88. Quorum voting – split votes
  • 89. Quorum voting – split votes
  • 90. Quorum voting – split votes
  • 91. Quorum voting – split votes
  • 92. Quorum voting – split votes
  • 93. James Mickens “The Saddest Moment” http://research.microsoft.com/en-us/people/mickens/thesaddestmoment.pdf
  • 94. Paxos
  • 95. Basic Paxos = “choose exactly one value”
  • 96. Paxos - photo by Luigi Piazzi
  • 97. Paxos: a high-level overview It’s the distributed systems algorithm
  • 98. Paxos: a high-level overview JavaZone had a full session on Paxos already today…
  • 99. A few Paxos whitepapers "Reaching Agreement in the Presence of Faults” – Lamport, 1980 … “FLP Impossibility Result” – Fisher et al, 1985 “The Part Time Parliament” – Lamport, 1998 … “Paxos made Simple” – Lamport, 2001 “Fast Paxos” – Lamport, 2005 … “Paxos made Live” – Chandra et al, 2007 … “Paxos made Moderately Complex” – Rennesse, 2011 ;-)
  • 107. ! Consensus time! Chose a value (raise your hand)
  • 108. Consensus time! Chose a value (raise your hand): v1 = Basic Paxos + Raft v2 = Just Raft
  • 109. Consensus time! Chose a value (raise your hand): v1 = Basic Paxos + Raft v2 = Just Raft
  • 110. Consensus time! Chose a value (raise your hand): v1 = Basic Paxos + Raft v2 = Just Raft
  • 111. Consensus time! Chose a value (raise your hand): v1 = Basic Paxos + Raft v2 = Just Raft (if enough time, Paxos)
  • 112. Basic Paxos simple example
  • 113. Paxos: Proposals ProposalNr must: • be greaterThan any prev proposalNr used by this Proposer • example: [roundNr|serverId]
  • 114. Paxos: 2 phases Phase 1: Prepare Phase 2: Accept
  • 115. Paxos, Prepare Phase n = nextSeqNr()
  • 116. Paxos, Prepare Phase acceptors ! Prepare(n, value)
  • 117. Paxos, Prepare Phase case Prepare(n, value) =>! if (n > minProposal) {! minProposal = n! accVal = value! }! ! sender() ! Accepted(minProposal, accVal)
  • 118. Paxos, Prepare Phase case Prepare(n, value) =>! if (n > minProposal) {! minProposal = n! accVal = value! }! ! sender() ! Accepted(minProposal, accVal)
  • 119. Paxos, Prepare Phase value = highestN(responses).accVal ! // replace my value, with accepted value!
  • 120. Paxos, Accept Phase acceptors ! Accept(n, value)
  • 121. Paxos, Accept Phase case Accept(n, value) =>! if (n >= minProposal) {! acceptedProposal = minProposal = n! acceptedValue = value! }! ! learners ! Learn(value)! sender() ! minProposal
  • 124. Paxos, Accept Phase if (acceptedN > n) restartPaxos()! else println(n + “ was chosen!”)
  • 125. Basic Paxos Basic Paxos, needs extensions for the “real world”. Additions: • “stable leader” • performance (basic = 2 * broadcast roundtrip) • ensure full replication • configuration changes
  • 127. Multi Paxos “Basically everyone does it, but everyone does it differently.”
  • 128. Multi Paxos • Keeps the Leader • Clients find and talk to the Leader • Skips Phase 1, in stable state • 2 delays instead of 4, until learning a value
  • 129. Raft
  • 130. Raft – inspired by Paxos Paxos is great. Multi-Paxos is great, but no “common understanding”. ! ! Raft wants to be understandable and just as solid. "In search of an understandable consensus protocol" (2013)
  • 131. Raft – inspired by Paxos ! ! • Leader based • Less processes than Paxos • It’s goal is simplicity • “Basic” includes snapshotting / membership
  • 132. Raft - summarised on one page Diego Ongaro & John Ouserhout – In search of an understandable consensus protocol
  • 133. Raft
  • 134. Raft
  • 135. Raft - starting the cluster
  • 136. Raft - Election timeout
  • 137. Raft - 1st election
  • 138. Raft - 1st election
  • 139. Raft - Election Timeout
  • 140. Raft - Election Phase
  • 141. Raft
  • 142. Raft
  • 143. Raft
  • 144. Raft
  • 145. Raft
  • 146. Raft
  • 147. Raft
  • 148. Raft
  • 149. Raft
  • 150. Raft
  • 151. Raft – heartbeat = empty entries
  • 152. Raft – heartbeat = empty entries
  • 153. Akka–Raft ! (community project) (work in progress)
  • 155. Raft translated to Akka abstract class RaftActor ! ! extends Actor ! ! with FSM[RaftState, Metadata]
  • 156. Raft translated to Akka abstract class RaftActor ! ! extends Actor ! ! with FSM[RaftState, Metadata]
  • 157. Raft translated to Akka onTransition {! ! case Follower -> Candidate =>! self ! BeginElection! resetElectionDeadline()! ! // ...! }
  • 158. Raft translated to Akka onTransition {! ! case Follower -> Candidate =>! self ! BeginElection! resetElectionDeadline()! ! // ...! }
  • 159. Raft translated to Akka ! case Event(BeginElection, m: ElectionMeta) =>! log.info("Init election (among {} nodes) for {}”,! m.config.members.size, m.currentTerm)! ! val request = RequestVote(m.currentTerm, m.clusterSelf, replicatedLog.lastTerm, replicatedLog.lastIndex)! ! m.membersExceptSelf foreach { _ ! request }! ! val includingThisVote = m.incVote! stay() using includingThisVote.withVoteFor(m.currentTerm, m.clusterSelf)! }!
  • 161. Raft Heartbeat using Akka sendHeartbeat(m)! log.info("Starting hearbeat, with interval: {}", heartbeatInterval)! setTimer(HeartbeatName, SendHeartbeat, heartInterval, repeat = true)! akka-raft is a work in progress community project – it may change a lot
  • 162. Raft Heartbeat using Akka sendHeartbeat(m)! log.info("Starting hearbeat, with interval: {}", heartbeatInterval)! setTimer(HeartbeatName, SendHeartbeat, heartInterval, repeat = true)! akka-raft is a work in progress community project – it may change a lot
  • 163. Raft Heartbeat using Akka sendHeartbeat(m)! log.info("Starting hearbeat, with interval: {}", heartbeatInterval)! setTimer(HeartbeatName, SendHeartbeat, heartInterval, repeat = true)! val leaderBehaviour = {! // ...! case Event(SendHeartbeat, m: LeaderMeta) =>! sendHeartbeat(m)! stay()! akka-raft is a work in progress community project – it may change a lot }
  • 164. Akka-Raft in User-Land //alpha!!! class WordConcatRaftActor extends RaftActor {! ! type Command = Cmnd! ! var words = Vector[String]()! ! /** Applied when command committed by Raft consensus */! def apply = {! case AppendWord(word) =>! words = words :+ word! word! ! case GetWords =>! log.info("Replying with {}", words.toList)! words.toList! }! }! akka-raft is a work in progress community project – it may change a lot
  • 166. FLP Impossibility Proof (19 Impossibility of Distributed Consensus with One Faulty Process 1985 by Fisher, Lynch, Paterson
  • 167. FLP Impossibility Result Impossibility of Distributed Consensus with One Faulty Process 1985 by Fisher, Lynch, Paterson
  • 168. FLP Impossibility Result Impossibility of Distributed Consensus with One Faulty Process 1985 by Fisher, Lynch, Paterson
  • 169. ktoso @ typesafe.com twitter: ktosopl github: ktoso blog: project13.pl team blog: letitcrash.com JavaZone @ Oslo 2014 ! ! Takk! Dzięki! Thanks! ありがとう! akka.io
  • 170. Happy Byzantine Lunch-time! Konrad 'ktoso' Malawski GeeCON 2014 @ Kraków, PL
  • 171. ©Typesafe 2014 – All Rights Reserved
  • 172. Links 1. http://cs-www.cs.yale.edu/homes/arvind/cs425/doc/fischer.pdf 2. http://hydra.infosys.tuwien.ac.at/teaching/courses/AdvancedDistributedSystems/download/ 1975_Akkoyunlu,%20Ekanadham,%20Huber_Some%20constraints%20and%20tradeoffs %20in%20the%20design%20of%20network%20communications.pdf 3. http://research.microsoft.com/en-us/people/mickens/thesaddestmoment.pdf 4. http://research.microsoft.com/en-us/um/people/lamport/pubs/lamport-paxos.pdf 5. http://research.microsoft.com/en-us/um/people/lamport/pubs/paxos-simple.pdf 6. http://the-paper-trail.org/blog/consensus-protocols-paxos/ 7. http://static.googleusercontent.com/media/research.google.com/en//archive/ paxos_made_live.pdf 8. http://static.googleusercontent.com/media/research.google.com/en//archive/chubby-osdi06. pdf 9. https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf 10. Recent Leslie Lamport interview: http://www.se-radio.net/2014/04/episode-203-leslie-lamport- on-distributed-systems/ 11. http://book.mixu.net/distsys/ 12. http://codahale.com/you-cant-sacrifice-partition-tolerance/ Peter Deutsch “The Eight Fallacies of Distributed Computing” https://blogs.oracle.com/jag/resource/Fallacies.html
  • 173. Links 1. Excellent Paxos lecture by Diego Ongaro https://www.youtube.com/watch?v=JEpsBg0AO6o 2. Fallacies, actual paper: http://www.rgoarchitects.com/Files/fallacies.pdf 3. Diego Ongaro & John Ouserhout – In search of an understandable consensus protocol 4. http://macs.citadel.edu/rudolphg/csci604/ImpossibilityofConsensus.pdf Peter Deutsch “The Eight Fallacies of Distributed Computing” https://blogs.oracle.com/jag/resource/Fallacies.html
  • 174. Images / drawings 1. Paxos Island Photo – Luigi Piazzi (CC license) https://www.flickr.com/photos/photolupi/ 3686769346/in/photolist-6BME5J-orKHL2-58qmez-58uz7s-7bRwTj-7bRvHY-6DdRC2- fBqFFU-35KTg7-8vbe23-bsBGL7-58qq6z-58uAjG-8vbeCd-d1Sqqw-d1Smsj-d1Sqi5- d1SoMA-d1SmBE-d1SpVo-d1Sk2U-d1SoBQ-d1SoXu-d1SoqN-d1Spqu-d1Sq4w-d1SpLU-d1SKDG- d1Skcu-d1Sp8f-d1Sqaq-d1SpCw-75YaVN-d1SLs1-d1SK15-d1SJiC-d1Suiu-d1SKtS-d1SjQS- d1StyU-d1SKi1-d1SxGS-d1Sm6j-d1Sxdh-d1SKMN-d1SxAq-d1SwgC-d1Smgj-d1SvhJ- d1SjC7 2. Drawings – myself (use-them-at-will-unless-mocking-my-horrible-drawing-skills-license) Peter Deutsch “The Eight Fallacies of Distributed Computing” https://blogs.oracle.com/jag/resource/Fallacies.html