Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Akka cluster overview at 010dev

6,072 views

Published on

Akka cluster overview at 010dev

  1. 1. Distributed by Design Dr. Roland Kuhn @rolandkuhnonsdag 3 april 13
  2. 2. outline actors remote actors rise of the cluster when the cluster grows up adding typesonsdag 3 april 13
  3. 3. What is an Actor? • Akkas unit of computation is called an Actor • Actors are purely reactive components: – a mailbox – behavior & state – scheduled to run when sent a message • Each actor has a parent, handling its failuresonsdag 3 april 13
  4. 4. Actor Behavior Stateonsdag 3 april 13
  5. 5. Event-driven Thread Actor Behavior Stateonsdag 3 april 13
  6. 6. Event-driven Thread Actor Behavior Stateonsdag 3 april 13
  7. 7. Event-driven Thread Actor Behavior Stateonsdag 3 april 13
  8. 8. Event-driven Thread Actor Behavior Stateonsdag 3 april 13
  9. 9. Actor Behavior Stateonsdag 3 april 13
  10. 10. Event-driven Thread Actor Behavior Stateonsdag 3 april 13
  11. 11. Define Actor public class Greeting implements Serializable { public final String who; public Greeting(String who) { this.who = who; } } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); } } }onsdag 3 april 13
  12. 12. Define Actor Define the message(s) the Actor should be able to respond to public class Greeting implements Serializable { public final String who; public Greeting(String who) { this.who = who; } } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); } } }onsdag 3 april 13
  13. 13. Define Actor Define the message(s) the Actor should be able to respond to public class Greeting implements Serializable { public final String who; public Greeting(String who)the Actor class = who; } Define { this.who } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); } } }onsdag 3 april 13
  14. 14. Define Actor Define the message(s) the Actor should be able to respond to public class Greeting implements Serializable { public final String who; public Greeting(String who)the Actor class = who; } Define { this.who } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); } } Define the Actor’s behavior }onsdag 3 april 13
  15. 15. Create Actor public class Greeting implements Serializable { public final String who; public Greeting(String who) { this.who = who; } } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); } } } ActorSystem system = ActorSystem.create("MySystem"); ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter");onsdag 3 april 13
  16. 16. Create Actor public class Greeting implements Serializable { public final String who; public Greeting(String who) { this.who = who; } } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); } Create an Actor system } } ActorSystem system = ActorSystem.create("MySystem"); ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter");onsdag 3 april 13
  17. 17. Create Actor public class Greeting implements Serializable { public final String who; public Greeting(String who) { this.who = who; } } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); Actor configuration } Create an Actor system } } ActorSystem system = ActorSystem.create("MySystem"); ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter");onsdag 3 april 13
  18. 18. Create Actor public class Greeting implements Serializable { public final String who; public Greeting(String who) { this.who = who; } } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); Actor configuration } Create an Actor system } } ActorSystem system = ActorSystem.create("MySystem"); ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter"); Give it a nameonsdag 3 april 13
  19. 19. Create Actor public class Greeting implements Serializable { public final String who; public Greeting(String who) { this.who = who; } } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); Actor configuration } Create an Actor system } } ActorSystem system = ActorSystem.create("MySystem"); ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter"); Create the Actor Give it a nameonsdag 3 april 13
  20. 20. Create Actor public class Greeting implements Serializable { public final String who; public Greeting(String who) { this.who = who; } } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); Actor configuration } Create an Actor system } } ActorSystem system = ActorSystem.create("MySystem"); ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter"); You get an ActorRef back Create the Actor Give it a nameonsdag 3 april 13
  21. 21. Actors can form hierarchies Guardian System Actoronsdag 3 april 13
  22. 22. Actors can form hierarchies Guardian System Actor system.actorOf( new Props(Foo.class), “Foo”);onsdag 3 april 13
  23. 23. Actors can form hierarchies Guardian System Actor system.actorOf( Foo new Props(Foo.class), “Foo”);onsdag 3 april 13
  24. 24. Actors can form hierarchies Guardian System Actor Foo getContext().actorOf( new Props(A.class), “A”);onsdag 3 april 13
  25. 25. Actors can form hierarchies Guardian System Actor Foo getContext().actorOf( A new Props(A.class), “A”);onsdag 3 april 13
  26. 26. Actors can form hierarchies Guardian System Actor Foo Bar A C A B E C B Donsdag 3 april 13
  27. 27. Name resolution - like a file-system Guardian System Actor Foo Bar A C A B E C B Donsdag 3 april 13
  28. 28. Name resolution - like a file-system Guardian System Actor /Foo Foo Bar A C A B E C B Donsdag 3 april 13
  29. 29. Name resolution - like a file-system Guardian System Actor /Foo Foo Bar /Foo/A A C A B E C B Donsdag 3 april 13
  30. 30. Name resolution - like a file-system Guardian System Actor /Foo Foo Bar /Foo/A A C A /Foo/A/B B E C B Donsdag 3 april 13
  31. 31. Name resolution - like a file-system Guardian System Actor /Foo Foo Bar /Foo/A A C A /Foo/A/B B E C B D /Foo/A/Donsdag 3 april 13
  32. 32. Send Message public class Greeting implements Serializable { public final String who; public Greeting(String who) { this.who = who; } } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); } } } ActorSystem system = ActorSystem.create("MySystem"); ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter"); greeter.tell(new Greeting("Charlie Parker"), null);onsdag 3 april 13
  33. 33. Send Message public class Greeting implements Serializable { public final String who; public Greeting(String who) { this.who = who; } } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); } } } ActorSystem system = ActorSystem.create("MySystem"); ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter"); greeter.tell(new Greeting("Charlie Parker"), null); Send the messageonsdag 3 april 13
  34. 34. Full example public class Greeting implements Serializable { public final String who; public Greeting(String who) { this.who = who; } } public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0; public void onReceive(Object message) { if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who); } } } ActorSystem system = ActorSystem.create("MySystem"); ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter"); greeter.tell(new Greeting("Charlie Parker"), null);onsdag 3 april 13
  35. 35. Distributable by Designonsdag 3 april 13
  36. 36. onsdag 3 april 13
  37. 37. Error Kernelonsdag 3 april 13
  38. 38. Node 1 Node 2 Error Kernelonsdag 3 april 13
  39. 39. Create Actor ActorRef greeter = system.actorOf(new Props( GreetingActor.class), "greeter");onsdag 3 april 13
  40. 40. Remote Deployment Just feed the ActorSystem with this configuration akka { actor { provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = } } } }onsdag 3 april 13
  41. 41. Remote Deployment Just feed the ActorSystem with this configuration Configure a Remote Provider akka { actor { provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = } } } }onsdag 3 april 13
  42. 42. Remote Deployment Just feed the ActorSystem with this configuration Configure a Remote Provider akka { For the Greeter actor { actor provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = } } } }onsdag 3 april 13
  43. 43. Remote Deployment Just feed the ActorSystem with this configuration Configure a Remote Provider akka { For the Greeter actor { actor provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = } } Define Remote Path } }onsdag 3 april 13
  44. 44. Remote Deployment Just feed the ActorSystem with this configuration Configure a Remote Provider akka { For the Greeter actor { actor provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = akka:// } } Define Remote Path } Protocol }onsdag 3 april 13
  45. 45. Remote Deployment Just feed the ActorSystem with this configuration Configure a Remote Provider akka { For the Greeter actor { actor provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = akka://MySystem } } Define Remote Path } Protocol Actor System }onsdag 3 april 13
  46. 46. Remote Deployment Just feed the ActorSystem with this configuration Configure a Remote Provider akka { For the Greeter actor { actor provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = akka://MySystem@machine1 } } Define Remote Path } Protocol Actor System Hostname }onsdag 3 april 13
  47. 47. Remote Deployment Just feed the ActorSystem with this configuration Configure a Remote Provider akka { For the Greeter actor { actor provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = akka://MySystem@machine1:2552 } } Define Remote Path } Protocol Actor System Hostname Port }onsdag 3 april 13
  48. 48. Remote Deployment Just feed the ActorSystem with this configuration Configure a Remote Provider akka { For the Greeter actor { actor provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = akka://MySystem@machine1:2552 } } Define Remote Path } Protocol Actor System Hostname Port } Zero code changesonsdag 3 april 13
  49. 49. Remote Lookup ActorRef greeter = system.actorFor( "akka://MySystem@machine1:2552/user/greeter" );onsdag 3 april 13
  50. 50. Can you see the problem?onsdag 3 april 13
  51. 51. Fixed Addresses akka { actor { provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = akka://MySystem@machine1:2552 } } } } ActorRef greeter = system.actorFor( "akka://MySystem@machine1:2552/user/greeter" );onsdag 3 april 13
  52. 52. Akka Clusteronsdag 3 april 13
  53. 53. Akka Cluster 2.1 • Gossip-based Cluster Membership • Failure Detector 2 .1 • Cluster DeathWatch in vi ew • Cluster-Aware Routers p re nt al m e ri expe er is lu st Consdag 3 april 13
  54. 54. Cluster Membership • Node ring à la Riak / Dynamo • Gossip-protocol for state dissemination • Vector Clocks to detect convergenceonsdag 3 april 13
  55. 55. Node ring with gossiping members Member Nodeonsdag 3 april 13
  56. 56. Node ring with gossiping members Member Node Member Member Node Node Member Member Node Node Member Member Node Node Member Member Node Node Member Nodeonsdag 3 april 13
  57. 57. Node ring with gossiping members Member Node Member Member Node Node Member Member Node Node Member Member Node Node Member Member Node Node Member Nodeonsdag 3 april 13
  58. 58. Node ring with gossiping members Member Node Member Member Node Node Member Member Node Node Member Member Node Node Member Member Node Node Member Nodeonsdag 3 april 13
  59. 59. Vector Clock • Vector Clocks are used to: - Generate a partial ordering of events in a distributed system - Detecting causality violations • We use Vector Clocks to to reconcile and merge differences in cluster stateonsdag 3 april 13
  60. 60. Gossiping Protocolonsdag 3 april 13
  61. 61. Gossiping Protocol Used for:onsdag 3 april 13
  62. 62. Gossiping Protocol Used for: – Cluster Membershiponsdag 3 april 13
  63. 63. Gossiping Protocol Used for: – Cluster Membership – Configuration dataonsdag 3 april 13
  64. 64. Gossiping Protocol Used for: – Cluster Membership – Configuration data – Leader Determinationonsdag 3 april 13
  65. 65. Gossiping Protocol Used for: – Cluster Membership – Configuration data – Leader Determination – Partitioning dataonsdag 3 april 13
  66. 66. Gossiping Protocol Used for: – Cluster Membership – Configuration data – Leader Determination – Partitioning data – Naming Serviceonsdag 3 april 13
  67. 67. Push/Pull Gossip • Push 2 .3+ in – sender only sends versions (Vector Clock) n za tio • Pull ti mi op – receivere u r only asks for information for which it has u toutdated version f an • Partly biased – send fraction of gossip to nodes with older stateonsdag 3 april 13
  68. 68. Cluster Convergenceonsdag 3 april 13
  69. 69. Cluster Convergence • When each Node has seen the same Vector Clockonsdag 3 april 13
  70. 70. Cluster Convergence • When each Node has seen the same Vector Clock • unreachable nodes will fail thisonsdag 3 april 13
  71. 71. Cluster Convergence • When each Node has seen the same Vector Clock • unreachable nodes will fail this • mark nodes DOWN to proceedonsdag 3 april 13
  72. 72. Cluster Convergence • When each Node has seen the same Vector Clock • unreachable nodes will fail this • mark nodes DOWN to proceed – manual Ops interventiononsdag 3 april 13
  73. 73. Cluster Convergence • When each Node has seen the same Vector Clock • unreachable nodes will fail this • mark nodes DOWN to proceed – manual Ops intervention – automatic actiononsdag 3 april 13
  74. 74. Member Statesonsdag 3 april 13
  75. 75. Member States • JOINING • UP • LEAVING • EXITING • DOWN • REMOVEDonsdag 3 april 13
  76. 76. Member States • JOINING • UP • LEAVING • EXITING • DOWN • REMOVEDonsdag 3 april 13
  77. 77. Leaderonsdag 3 april 13
  78. 78. Leader • Any node can be the leaderonsdag 3 april 13
  79. 79. Leader • Any node can be the leader • Just takes the role of being a leaderonsdag 3 april 13
  80. 80. Leader • Any node can be the leader • Just takes the role of being a leader • Is deterministically recognized by all nodesonsdag 3 april 13
  81. 81. Leader • Any node can be the leader • Just takes the role of being a leader • Is deterministically recognized by all nodes – always the first member in the sorted membership ringonsdag 3 april 13
  82. 82. Cluster Events public class Listener extends UntypedActor { public void onReceive(Object message) { if (message instanceof MemberUp) { // ... } } } ActorRef listener = system.actorOf(new Props(Listener.class), "listener"); Cluster.get(system).subscribe(listener, MemberEvent.class);onsdag 3 april 13
  83. 83. Cluster Events public class Listener extends UntypedActor { public void onReceive(Object message) { if (message instanceof MemberUp) { MemberUp mUp = (MemberUp) message; getContext().actorFor(mUp.address() + "/user/greeter").tell( new Greeting("Charlie Parker"), getSelf()); } } } ActorRef listener = system.actorOf(new Props(Listener.class), "listener"); Cluster.get(system).subscribe(listener, MemberEvent.class);onsdag 3 april 13
  84. 84. Phi Accrual Failure Detector regular messages A B • B monitors A • Sample inter-arrival time to expect next beat • B measures continuum of deadness of A http://ddg.jaist.ac.jp/pub/HDY+04.pdfonsdag 3 april 13
  85. 85. Phi Accrual Failure Detector regular messages A B • B monitors A • Sample inter-arrival time to expect next beat • B measures continuum of deadness of A http://ddg.jaist.ac.jp/pub/HDY+04.pdfonsdag 3 april 13
  86. 86. Selective Failure Detection Member Node Member Member Node Node Member Member Node Node Heartbeat Member Member Node Node Member Member Node Member Node Nodeonsdag 3 april 13
  87. 87. Selective Failure Detection Member Node Member Member Node Node Member Member Node Node Heartbeat Member Member Node Node Member Member Node Member Node Nodeonsdag 3 april 13
  88. 88. Cluster DeathWatch • Triggered by marking node «A» DOWN – Tell parents of their lost children on «A» – Kill all children of actors on «A» – Send Terminated for actors on «A»onsdag 3 april 13
  89. 89. Enable clustering akka { actor { provider = "akka.cluster.ClusterActorRefProvider" ... }   extensions = ["akka.cluster.Cluster"]   cluster { seed-nodes = [ "akka://ClusterSystem@127.0.0.1:2551", "akka://ClusterSystem@127.0.0.1:2552" ] } }onsdag 3 april 13
  90. 90. Load Balancingonsdag 3 april 13
  91. 91. Routers ActorRef routerActor = getContext().actorOf( new Props(ExampleActor.class). withRouter(new RoundRobinRouter(nrOfInstances)) );onsdag 3 april 13
  92. 92. …or from config akka.actor.deployment { /path/to/actor { router = round-robin nr-of-instances = 5 } }onsdag 3 april 13
  93. 93. Configure a clustered router akka.actor.deployment { /statsService/workerRouter { router = consistent-hashing nr-of-instances = 100 cluster { enabled = on max-nr-of-instances-per-node = 3 allow-local-routees = on } } }onsdag 3 april 13
  94. 94. Multi Node Testing object MultiNodeSampleConfig extends MultiNodeConfig { val node1 = role("node1") val node2 = role("node2") } "A MultiNodeSample" must { "wait for all nodes to enter a barrier" in { enterBarrier("startup") } }onsdag 3 april 13
  95. 95. Multi Node Testing runOn(node2) { system.actorOf(Props(new Actor { def receive = { case "ping" => sender ! "pong" } }), "ponger") } enterBarrier("deployed") runOn(node1) { val ponger = system.actorFor(node(node2) / "user" / "ponger") ponger ! "ping" expectMsg("pong") } enterBarrier("finished")onsdag 3 april 13
  96. 96. … when the Cluster grows uponsdag 3 april 13
  97. 97. Adaptive Load Balancing • Metrics collected and spread – Heap memory – CPU, system load • Adaptive Router – Biased random with weights based on capacityonsdag 3 april 13
  98. 98. One tree to rule them all • One Actor tree per node • Cluster tree is mapped to local sub-treesonsdag 3 april 13
  99. 99. One tree to rule them allonsdag 3 april 13
  100. 100. One tree to rule them allonsdag 3 april 13
  101. 101. One tree to rule them allonsdag 3 april 13
  102. 102. The Magic Sauce • User code only sees cluster://... names • ActorRef becomes repointable – local – remote • Can now move actors around transparently – Actor encapsulation makes it possibleonsdag 3 april 13
  103. 103. What does this enable? • Actor migration • Actor replication • Automatic cluster partitioning – later also based on runtime metrics • Node fail-over – first for stateless actors – later for stateful actors using event sourcing ➾ Fault Tolerance & Distributiononsdag 3 april 13
  104. 104. Typed Channels Experimental in 2.2onsdag 3 april 13
  105. 105. The Problem someActor ! CommandOneonsdag 3 april 13
  106. 106. The Problem trait Command case class CommandOne(param: String) extends Command someActor ! CommandOneonsdag 3 april 13
  107. 107. The Vision someActor <-!- CommandOne(”msg”) because the other does not compileonsdag 3 april 13
  108. 108. But How? • ActorRef must know about message types – Actor type must be parameterized • Message type is verified against thatonsdag 3 april 13
  109. 109. And the replies? val f: Future[Response] = someActor <-?- CommandOne(”hello”) because the compiler knowsonsdag 3 april 13
  110. 110. And How This? • ActorRef must know reply types – Actor must be parameterized with them • Reply types are extracted at call siteonsdag 3 april 13
  111. 111. No Type Pollution • Generic Filter/Transform Actors – accept management commands – pass on generic other type • Using just one type is not enough! • Need to use type unions and allow multiple possible reply types for one inputonsdag 3 april 13
  112. 112. The Implementation • Tagged type union with :+:[(In, Out), ChannelList] <: ChannelList • Value class ChannelRef[…](val a: ActorRef) • Actor mixin Channels[…] • WrappedMessage[…, LUB](val m: LUB) • ops desugar to tell/ask after type checkonsdag 3 april 13
  113. 113. Actors Do Compose msg -?-> firstActor -?-> secondActor -!-> client msg -?-> someService -*-> (_ map httpOk) -!-> client Process wiring from the outsideonsdag 3 april 13
  114. 114. How to Declare it? class OpinionatedEcho extends Actor with Channels[TNil, (String, String) :+: TNil] { channel[String] { (str, sender) ⇒ sender <-!- str } // or channel[String] { case (”hello”, sender) ⇒ sender <-!- ”world” case (x, sender) ⇒ sender <-!- s”dunno: $x” } } “sender” will accept only String messagesonsdag 3 april 13
  115. 115. The Result:onsdag 3 april 13
  116. 116. Type-Safe Composability of Actor Systemsonsdag 3 april 13
  117. 117. get it and learn more http://akka.io http://letitcrash.com http://typesafe.comonsdag 3 april 13
  118. 118. E0Fonsdag 3 april 13

×