Effective akka scalaio

1. Effective Actors Jamie Allen Director of Consulting @jamie_allen

2. @jamie_allen 2

3. Goal Communicate as much about what I’ve learned in 4+ years of actor development within one hour NOTE -‐ this is not an introductory talk, and will contain some advanced concepts about Akka & asynchronous development @jamie_allen 3

4. Extra and Cameo Patterns A couple of patterns for how to handle individual messages to an actor without making it do all of the work before handling the next message @jamie_allen 4

5. Request Aggregation •When an actor handles a message, it frequently has to perform multiple interactions with other services to provide a valid response •We do not want the actor that received the message to be tied up performing that work •What are our options? @jamie_allen 5

6. Use Case •An actor will receive a request to get all of the account balances for a customer (savings, checkings and money market) •Actor should not wait to ﬁnish handling one request before handling another •Actor should receive service proxies from which it can retrieve each account’s info •Actor should either get responses from all three within a speciﬁed time, or send a timeout response back to the requestor @jamie_allen 6

7. Request Aggregation Message 4 Message 3 Message 1 Handler Message 2 Message 1 Message 2 Handler Request Receiver Message 3 Handler Message 4 Handler @jamie_allen 7

8. Transactions? •Could be! •You have to write the logic of how to roll back if anything fails •But you have the control and the context to do it, especially if your eﬀects are going to multiple external places or data stores @jamie_allen 8

9. All Code is on GitHub • I’m going to be showing some reasonably complex examples • Don’t worry about trying to copy the code from the slides http://github.com/jamie-allen/effective_akka @jamie_allen 9

10. Use Futures and Promises? •No! Use another actor responsible for: •Capturing the context (original requestor) •Deﬁning how to handle responses from other services •Deﬁning the timeout that will race against your •Each future and the resulting promise have an additional cost that we do not need to pay @jamie_allen 10

11. Use Futures and Promises? def receive = { case GetCustomerAccountBalances(id) => val futSavings = savingsAccounts ? GetCustomerAccountBalances(id) val futChecking = checkingAccounts ? GetCustomerAccountBalances(id) val futMM = moneyMarketAccounts ? GetCustomerAccountBalances(id) Yuck! val futBalances = for { savings <- futSavings.mapTo[Option[List[(Long, BigDecimal)]]] checking <- futChecking.mapTo[Option[List[(Long, BigDecimal)]]] mm <- futMM.mapTo[Option[List[(Long, BigDecimal)]]] } yield AccountBalances(savings, checking, mm) futBalances map (sender ! _) } } @jamie_allen 11

12. Capturing the Sender •This is trickier than it sounds •You need the “sender” value from the actor that received the original request, not the sender inside of the actor handling it! •One of the biggest sources of actor bugs @jamie_allen 12

13. Use Futures and Promises? def receive = { case GetCustomerAccountBalances(id) => val futSavings = savingsAccounts ? GetCustomerAccountBalances(id) val futChecking = checkingAccounts ? GetCustomerAccountBalances(id) val futMM = moneyMarketAccounts ? GetCustomerAccountBalances(id) val futBalances = for { savings <- futSavings.mapTo[Option[List[(Long, BigDecimal)]]] checking <- futChecking.mapTo[Option[List[(Long, BigDecimal)]]] mm <- futMM.mapTo[Option[List[(Long, BigDecimal)]]] } yield AccountBalances(savings, checking, mm) futBalances map (sender ! _) } Bug! @jamie_allen 13

14. Use an Anonymous Actor? class OuterActor extends Actor def receive = LoggingReceive { case DoWork => { val originalSender = sender context.actorOf(Props(new Actor() { def receive = LoggingReceive { case HandleResponse(value) => timeoutMessager.cancel sendResponseAndShutdown(Response(value)) case WorkTimeout => sendResponseAndShutdown(WorkTimeout) } } Anonymous Actor def sendResponseAndShutdown(response: Any) = { originalSender ! response context.stop(self) } someService ! DoWork import context.dispatcher val timeoutMessager = context.system.scheduler.scheduleOnce(250 milliseconds) { self ! WorkTimeout } })) } } } @jamie_allen 14

15. Use an Anonymous Actor? •I call this the “Extra” Pattern •It’s not bad, but it has drawbacks: •Poor stack traces due to “name mangled” actor names, like $a •More diﬃcult to maintain the cluttered code, and developers have to read through the body of the anonymous actor to ﬁgure out what it is doing •More likely to “close over” state @jamie_allen 15

16. Anonymous Actor Example class AccountBalanceRetrieverFinal(savingsAccounts: ActorRef, checkingAccounts: ActorRef, moneyMarketAccounts: ActorRef) extends Actor with ActorLogging def receive = LoggingReceive { case GetCustomerAccountBalances(id) => { val originalSender = sender context.actorOf(Props(new Actor() { var checkingBalances, savingsBalances, mmBalances: Option[List[(Long, BigDecimal)]] = None def receive = LoggingReceive { case CheckingAccountBalances(balances) => checkingBalances = balances collectBalances case SavingsAccountBalances(balances) => savingsBalances = balances collectBalances case MoneyMarketAccountBalances(balances) => mmBalances = balances collectBalances case AccountRetrievalTimeout => sendResponseAndShutdown(AccountRetrievalTimeout) } def collectBalances = (checkingBalances, savingsBalances, mmBalances) match { case (Some(c), Some(s), Some(m)) => timeoutMessager.cancel sendResponseAndShutdown(AccountBalances(checkingBalances, savingsBalances, mmBalances)) case _ => } def sendResponseAndShutdown(response: Any) = { originalSender ! response context.stop(self) } savingsAccounts ! GetCustomerAccountBalances(id) checkingAccounts ! GetCustomerAccountBalances(id) moneyMarketAccounts ! GetCustomerAccountBalances(id) import context.dispatcher val timeoutMessager = context.system.scheduler.scheduleOnce(250 milliseconds) { self ! AccountRetrievalTimeout } })) } } @jamie_allen 16

17. Create a “Cameo” Actor •Externalize the behavior of such an anonymous actor into a speciﬁc type •Anyone maintaining the code now has a type name from which they can infer what the actor is doing •Most importantly, you can’t close over state from an enclosing actor -‐ it must be passed explicitly @jamie_allen 17

18. Create a “Cameo” Actor class WorkerActor extends Actor { def receive = LoggingReceive { case HandleResponse(value) => timeoutMessager.cancel sendResponseAndShutdown(Response(value)) case WorkTimeout => sendResponseAndShutdown(WorkTimeout) } def sendResponseAndShutdown(response: Any) = { originalSender ! response context.stop(self) } import context.dispatcher val timeoutMessager = context.system.scheduler.scheduleOnce(250 milliseconds) { self ! WorkTimeout } } class DelegatingActor extends Actor def receive = LoggingReceive { case DoWork => { val originalSender = sender val worker = context.actorOf(WorkerActor.props(), “worker”) someService.tell(DoWork, worker) } } } @jamie_allen 18

19. Remember to Stop the Actor • When you are ﬁnished handling a request, ensure that the actor used is shutdown • This is a big memory leak if you don’t def sendResponseAndShutdown(response: Any) = { originalSender ! response log.debug("Stopping context capturing actor") context.stop(self) } @jamie_allen 19

20. Write Tests! •Always remember to write tests with your actors •Create unit tests that check the functionality of method calls without actor interations using TestActorRef •Create integration tests that send messages to actors and check the aggregated results @jamie_allen 20

21. Write Tests! class CameoSpec extends TestKit(ActorSystem("cameo-test-as")) with ImplicitSender with WordSpecLike with MustMatchers { val checkingAccountsProxy = system.actorOf(Props[CheckingAccountsProxyStub], "checkings") val moneyMarketAccountsProxy = system.actorOf(Props[MoneyMarketAccountsProxyStub], "money-markets") "An AccountBalanceRetriever" should { "return a list of account balances" in { val probe1 = TestProbe() val probe2 = TestProbe() val savingsAccountsProxy = system.actorOf(Props[SavingsAccountsProxyStub], "cameo-savings") val checkingAccountsProxy = system.actorOf(Props[CheckingAccountsProxyStub], "cameo-checkings") val moneyMarketAccountsProxy = system.actorOf(Props[MoneyMarketAccountsProxyStub], "cameo-mm") val accountBalanceRetriever = system.actorOf(Props(new AccountBalanceRetriever(savingsAccountsProxy, checkingAccountsProxy, moneyMarketAccountsProxy)), "cameo-retriever1") within(300 milliseconds) { probe1.send(accountBalanceRetriever, GetCustomerAccountBalances(1L)) val result = probe1.expectMsgType[AccountBalances] result must equal(AccountBalances(Some(List((3, 15000))), Some(List((1, 150000), (2, 29000))), Some(List()))) } within(300 milliseconds) { probe2.send(accountBalanceRetriever, GetCustomerAccountBalances(2L)) val result = probe2.expectMsgType[AccountBalances] result must equal(AccountBalances( Some(List((6, 640000), (7, 1125000), (8, 40000))), Some(List((5, 80000))), Some(List((9, 640000), (10, 1125000), (11, 40000))))) } } } } @jamie_allen 21

22. Write Tests! within(300 milliseconds) { probe1.send(accountBalanceRetriever, GetCustomerAccountBalances(1L)) val result = probe1.expectMsgType[AccountBalances] result must equal(AccountBalances( Some(List((3, 15000))), Some(List((1, 150000), (2, 29000))), Some(List()))) } within(300 milliseconds) { probe2.send(accountBalanceRetriever, GetCustomerAccountBalances(2L)) val result = probe2.expectMsgType[AccountBalances] result must equal(AccountBalances( Some(List((6, 640000), (7, 1125000), (8, 40000))), Some(List((5, 80000))), Some(List((9, 640000), (10, 1125000), (11, 40000))))) } @jamie_allen 22

23. Write Moar Tests! "return a TimeoutException when timeout is exceeded" in { val checkingAccountsProxy = system.actorOf(Props[CheckingAccountsProxyStub], "cameo-timing-out-checkings") within(250 milliseconds, 500 milliseconds) { probe.send(accountBalanceRetriever, GetCustomerAccountBalances(1L)) probe.expectMsg(AccountRetrievalTimeout) } } @jamie_allen 23

24. Best Practices Several important learnings I’ve gleaned in my years of actor development @jamie_allen 24

25. Create a Props Factory • Creating an actor within another actor implicitly closes over “this” • Necessary until spores (SIP-‐21) are part of Scala • Create a Props factory in a companion object • Anonymous actors can’t have one! object AccountBalanceResponseHandler { def props(savingsAccounts: ActorRef, checkingAccounts: ActorRef, moneyMarketAccounts: ActorRef, originalSender: ActorRef): Props = { Props(new AccountBalanceResponseHandler(savingsAccounts, checkingAccounts, moneyMarketAccounts, originalSender)) } } @jamie_allen 25

26. Keep Your Actors Simple • Do not conflate responsibilities in actors • Becomes hard to define the boundaries of responsibility • Supervision becomes more difficult as you handle more possibilities • Debugging becomes very difficult @jamie_allen 26

27. Be Explicit in Supervision • Every non-‐leaf node is technically a supervisor • Create explicit supervisors under each node for each type of child to be managed @jamie_allen 27

28. Conﬂated Supervision Customers C1 C2 Ac1 D1 Ac2 D2 Accounts & Devices D3 Applications A1 @jamie_allen A2 A3 A4 28

29. Explicit Supervision C1 C2 Customers Ac1 Accounts & Devices DS AS D1 Ac2 A1 @jamie_allen D2 A2 D3 A3 A4 Applications 29

30. Use Failure Zones • Multiple isolated zones with their own resources (thread pools, etc) • Prevents starvation of actors • Prevents issues in one branch from aﬀecting another val responseHandler = system.actorOf( AccountBalanceResponseHandler.props(), "cameo-handler").withDispatcher( "handler-dispatcher") @jamie_allen 30

31. No Failure Zones C1 C2 Customers DS AS Accounts & Devices Ac1 D1 Ac2 A1 @jamie_allen D2 A2 D3 A3 A4 Applications 31

32. Explicit Failure Zones C1 C2 Customers DS AS Accounts & Devices Ac1 D1 Ac2 A1 @jamie_allen D2 A2 D3 A3 A4 Applications 32

33. Push, Don’t Pull • Start with no guarantees about delivery • Add guarantees only where you need them • Retry until you get the answer you expect • Switch your actor to a "nominal" state at that point @jamie_allen 33

34. Beware the Thundering Herd • Actor systems can be overwhelmed by "storms" of messages ﬂying about • Do not pass generic messages that apply to many actors • Dampen actor messages if the exact same message is being handled repeatedly within a certain timeframe • Tune your dispatchers and mailboxes via back-‐oﬀ policies and queue sizes • Akka has Circuit Breakers to help you provide back-‐ pressure @jamie_allen 34

35. Create Granular Messages • Non-‐speciﬁc messages about general events are dangerous AccountsUpdated • Can result in "event storms" as all actors react to them • Use speciﬁc messages forwarded to actors for handling AccountDeviceAdded(acctNum, deviceNum) @jamie_allen 35

36. Unique IDs for Messages • Allows you to track message flow • When you find a problem, get the ID of the message that led to it • Use the ID to grep your logs and display output just for that message flow • Akka ensures ordering on a per actor basis, also in logging @jamie_allen 36

37. Create Specialized Exceptions • Don't use java.lang.Exception to represent failure in an actor • Speciﬁc exceptions can be handled explicitly • State can be transferred between actor incarnations in Akka (if need be) @jamie_allen 37

38. Never Reference “this” • Actors die • Doesn't prevent someone from calling into an actor with another thread • Akka solves this with the ActorRef abstraction • Never expose/publish “this” • Loop by sending messages to “self” • Register by sending references to your “self” @jamie_allen 38

39. Immutable Interactions •All messages passed between actors should be immutable •All mutable data to be passed with messages should be copied before sending •You don’t want to accidentally expose the very state you’re trying to protect @jamie_allen 39

40. Externalize Logic • Consider using external functions to encapsulate complex business logic • Easier to unit test outside of actor context • Not a rule of thumb, but something to consider as complexity increases • Not as big of an issue with Akka's TestKit @jamie_allen 40

41. Semantically Useful Logging • Trace-‐level logs should have output that you can read easily • Use line breaks and indentation • Both Akka and Erlang support hooking in multiple listeners to the event log stream @jamie_allen 41

42. Use the Typesafe Console in Development •Visual representations of actors at runtime are invaluable tools •Keep an eye out for actors whose mailboxes never drain and keep getting bigger •Look out for message handling latency that keeps going higher •These are signs that the actors cannot handle their load •Optimize with routers •Rethink usage of Dispatchers @jamie_allen 42

43. Use the Typesafe Console in Development @jamie_allen 43

44. Thank You! • Some content provided by members of the Typesafe team, including: • Jonas Bonér • Viktor Klang • Roland Kuhn • Havoc Pennington @jamie_allen 44

Effective akka scalaio

Effective akka scalaio

Recommended

More Related Content

Viewers also liked

Viewers also liked(20)

Similar to Effective akka scalaio

Similar to Effective akka scalaio(20)

More from shinolajla

More from shinolajla(16)

Recently uploaded

Recently uploaded(20)

Effective akka scalaio