The document discusses the problems with using Akka actors and proposes solutions using FSM and futures instead of actors for some cases. It describes issues with actors like lack of type safety, mutable state leading to complex code, and difficult debugging. FSMs are presented as a better solution by enforcing state transitions and logging. Futures are also suggested for cases without need for location transparency or real-time data flow that actors provide. The conclusion is that Akka requires a new mindset and can cause headaches if overused, so it's best to only use actors when truly needed.

1. The dark side of Akka
and the remedy
Ákos Kriváchy

2. Introduction - Ákos Kriváchy
• Scala disciple/convert/fanatic since 2013 February
• First FP language
• Akka since 2014 February
• Things I love about Scala:
• Static typing
• Partial Functions
• foldLeft, tail recursion
2

3. Recap on Akka
• Akka in key points:
• Messaging
• Actors
• Mailboxes (ActorRefs)
• Hierarchy
• Supervision
• Location transparency
Source: http://www.scottlogic.com/blog/2014/08/15/using-akka-and-scala-to-render-a-mandelbrot-set.html
3

4. Example from previous Meetup: URL scraping
4

5. To Akka or not to Akka?
• Akka solves:
• Concurrency
• Scalability and distributability
• Resilience
• Akka in our team:
•We had to rewrite a legacy Java component
• Chose Akka to try it out (after 4 months of research)
5

6. Problem #1: Any and Actor Ref
• All messages are Any-s
• Anything that’s not
handled ends up as a “dead
letter”
• Essentially a loss of all
typesafety
• Requires extensive testing
to “feel safe”
class MyActor(databasePersistence: ActorRef,
emailSender: ActorRef,
MQSender: ActorRef,
widgetDao: ActorRef,
twitterService: ActorRef)
extends Actor {
def receive: Receive = {
case Message(data) =>
twitterService ! Tweet(data)
// ...
case OtherMessage(_) =>
// ...
}
}
}
6

7. Solution: Typed Channels
val channelA: ChannelRef[(MsgA, MsgB) :+: TNil] = ???
val a = new MsgA
channelA <-!- a // send a to channelA
a -!-> channelA // same thing as above
val fB: Future[MsgB] = channelA <-?- a // ask the actor
Instead of the current solution:
val fB: Future[Any] = actorRef ? a
http://doc.akka.io/docs/akka/2.2.0/scala/typed-channels.html 7

8. NOPE!
http://doc.akka.io/docs/akka/2.3.0-RC1/project/migration-guide-2.2.x-2.3.x.html
8

9. It’s on the roadmap
9
August 28, 2014: http://typesafe.com/blog/akka-roadmap-update-2014
Lastly, we dare a look beyond the next major release: after
Akka 2.4.0 [early 2015] we will tackle the number one feature
request, namely more type-safety for inter-Actor messaging.
This will be a major research effort and likely also a disruptive
API change. For this reason Akka 2.4 is planned to be a long-term
support release, because even though we will try to find
ways for compatibility or at least an easy migration path we
cannot currently foresee the exact nature of all related
changes.

10. In the meantime: Don’t use Actors for everything
• What do we use instead?
• Scala Futures are extremely powerful
• Futures compose more nicely than Actors
• When should we use Actors?
• State in the application
• A cache is not state
• Extreme scaling is needed (location transparency)
• Need the flexibility to place any operation on any JVM -> use Actors for everything
• Realtime data flow applications (not request/response)
• What about supervision and „let it crash”?
• Future’s handle failure also
10

11. Code: https://github.com/krivachy/AkkaWithFsm
11

12. class Getter(url: String, depth: Int) extends Actor with ActorLogging with LinkParser {
lazy val client = new NingWSClient(new Builder().build()).url(url)
override def preStart() = {
client.get().map(_.body).map(Response).pipeTo(self)
}
def receive = {
case Response(body) =>
val links = parseLinks(body).toList
log.info(s"URL $url at depth $depth had ${links.size} links.")
links.foreach {
link =>
log.info(s"Sending link '$link'")
context.parent ! Controller.Check(link, depth)
}
context.stop(self)
case Status.Failure(cause) =>
log.error(s"Failed to GET $url", cause)
context.stop(self)
}
}
12

13. Simplified with Futures:
object Getter extends LinkParser {
case class Urls(urls: Set[String], depth: Int)
lazy val client = new NingWSClient(new Builder().build())
def get(url: String, depth: Int)(implicit ec: ExecutionContext): Future[Urls] = {
client.url(url).get().map {
response =>
Urls(parseLinks(response.body).toSet, depth)
}
}
}
13

14. Problem #2: Hellish code complexity
14

15. var hell
• Actors have too much mutable state
• Our worst scenario: Actor with 300 lines and 20 vars
• Hard to reason about state when everything is “global” inside the
Actor
• How do you initialize state that is only used in some cases?
• var something: SomeType = _
• NPE
• var something: Option[SomeType] = None
• Always need to “getOrElse”
15

16. become/unbecome hell
• Pushing and popping state on a stack
• context.become(behavior: Receive, discardOld: Boolean = true)
• context.unbecome()
• “context.become” isn’t enforced to be called last
• You use methods to keep things short, but there will be multiple methods
trying to modify the behaviour
• i.e. you could end up inadvertently overwriting behavior
• One place: context.become(handleCoolMessages orElse waitForNewRequests)
• Somewhere else: context.become(waitForNewRequests, discardOld = true)
• When things blow up you have no idea how you got there
16

17. class Receptionist extends Actor {
def receive = waiting
def waiting: Receive = {
case Api.Scrape(url, depth) =>
context.become(next(Vector(Job(sender, url, depth))))
}
def running(queue: Vector[Job]): Receive = LoggingReceive {
case Result(links) =>
// …
context.become(next(queue.tail))
case Api.Scrape(url, depth) =>
context.become(enqueue(queue, Job(sender, url, depth)))
case Terminated(_) =>
// …
context.become(next(queue.tail))
}
def enqueue(queue: Vector[Job], job: Job): Receive = LoggingReceive {
// …
running(queue)
}
def next(queue: Vector[Job]): Receive = LoggingReceive {
// …
running(queue)
}
}
17

18. Debug hell
• Debugging Actors is hard
• Stacktraces lack meaning
• Need to have a lot of boilerplate utility code:
• What was the message?
• Who sent the message?
• What’s my internal state?
• Possible solution:
• always override def preRestart(reason: Throwable, message: Option[Any])
• Still don’t know how we ended up in that become/unbecome state.
18

19. [ERROR] [akka://simple-actor/user/receptionist/controller-1/$a] Exception happened
java.lang.Exception: exception happened here
at meetup.akka.simple.Getter$$anonfun$receive$1$$anonfun$1.apply(Getter.scala:37)
at meetup.akka.simple.Getter$$anonfun$receive$1$$anonfun$1.apply(Getter.scala:37)
at scala.util.Try$.apply(Try.scala:191)
at meetup.akka.simple.Getter$$anonfun$receive$1.applyOrElse(Getter.scala:37)
at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
at meetup.akka.simple.Getter.aroundReceive(Getter.scala:16)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
at akka.dispatch.Mailbox.run(Mailbox.scala:220)
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
19

20. We need Logging for Debugging
• Akka provides:
• with ActorLogging
• log.info(…)
• LoggingRecieve: def receive = LoggingReceive { … }
• turn on: akka.actor.debug.receive=true
• Lifecycle logging: akka.actor.debug.lifecycle=true
• For all Actors
• Start, restart, stop, supervise, watch events
• Autorecieve logging: akka.actor.debug.autoreceive=true
• For all Actors
• Automatically handled messages: Stop, Kill, PoisionPill, Terminated, etc.
• Issue:
• Akka only provides logging of messages per Receive block (has pros and cons)
• If you missed one => good luck debugging issues around it in production
20

21. Logging example
[DEBUG] [akka://simple-actor/user/receptionist] started (meetup.akka.simple.Receptionist@14ba772)
[DEBUG] [akka://simple-actor/user/receptionist/controller-1] started (meetup.akka.simple.Controller@38a5d7)
[DEBUG] [akka://simple-actor/user/receptionist] now supervising Actor[akka://simple-actor/user/receptionist/controller-1#4232237]
[DEBUG] [akka://simple-actor/user/receptionist/controller-1] now watched by Actor[akka://simple-actor/user/receptionist#1565954732]
[DEBUG] [akka://simple-actor/user/receptionist/controller-1] received handled message
Check(http://doc.akka.io/docs/akka/2.3.5/intro/what-is-akka.html,1)
21

22. FSM
22

23. “Everything* is a Finite State Machine.”**
-me
*not everything
** do not quote me
23

24. FSM concepts
object Receptionist {
object Internal {
sealed trait State
case object Sleeping extends State
case object Processing extends State
sealed trait Data
case class NoQueue(requestId: Int = 0) extends Data
case class Queue(currentRequestId: Int, items: Vector[Job]) extends Data
}
}
class Receptionist extends FSM[Internal.State, Internal.Data] { … }
24

25. Define handlers for all states
// Initialize with data
startWith(Sleeping, NoQueue())
// Handlers for states
when(Sleeping)(enqueueNewRequest)
when(Processing) (processResult orElse enqueueNewRequest orElse reportError)
25

26. Enqueue New Request State Function
def enqueueNewRequest: StateFunction = {
case Event(Api.Scrape(url, depth), NoQueue(requestId)) =>
startControllerFor(requestId + 1, Vector(Job(sender(), url, depth)))
case Event(Api.Scrape(url, depth), queue: Queue) =>
if (queue.items.size > 3) {
stay replying Api.Failed(url)
} else {
goto(Processing) using
Queue(queue.currentRequestId, queue.items :+ Job(sender(), url, depth))
}
}
• Important:
• StateFunction: Event => State
• Event(incomingMessage, data) =>
• State transition: goto/stay (nextState) using (data) forMax(timeout) replying (message)
26

27. Monitoring state transitions
Internal:
onTransition {
case Idle -> Active => setTimer("timeout", Tick, 1 second, true)
case Active -> _ => cancelTimer("timeout")
case x -> Idle => log.info("entering Idle from " + x)
}
External:
monitoredActor ! SubscribeTransitionCallBack(self)
def recieve = {
case Transition(monitoredActor, oldState, newState) =>
if (newState == Errored) alert.raiseAlert(...)
}
override def postStop() = {
monitoredActor ! UnsubscribeTransitionCallBack(self)
}
27

28. Handling failure
whenUnhandled {
case Event(any, data) =>
val logUpToHere = prettyPrint(getLog)
log.error(s"Unhandled event: ${any}n$logUpToHere")
stay()
}
def failure: StateFunction = {
case Event(Status.Failure(cause), _) =>
log.error(s"Failed to GET $url", cause)
stop(FSM.Failure(cause))
}
onTermination {
case StopEvent(FSM.Normal, state, data) => ???
case StopEvent(FSM.Shutdown, state, data) => ???
case StopEvent(FSM.Failure(cause), state, data) => ???
} 28

29. Result is simplicity
class Receptionist extends FSM[Internal.State, Internal.Data] {
startWith(Sleeping, NoQueue())
when(Sleeping)(enqueueNewRequest)
when(Processing) (processResult orElse enqueueNewRequest orElse reportError)
def enqueueNewRequest: StateFunction = ???
def processResult: StateFunction = ???
def reportError: StateFunction = ???
private def nextQueueItem(queue: Queue): State = ???
private def startControllerFor(requestId: Int, queue: Vector[Job]): State = ???
whenUnhandled { ??? }
initialize()
}
29

30. Hell status
var ✔
No more mutable global state
inside Actors. Everything is
typed to the specific State.
become/unbecome ✔
All methods have to end in a
state transition. States are
clearly defined what they do.
debug/logging ?
30

31. LoggingFSM
• Remembers state transitions:
• override def logDepth = 8
• def getLog: Seq[LogEntry[Internal.State, Internal.Data]]
• Use with: onTermination
• Debug logging: akka.actor.debug.fsm=true
• Automatically logs important Events: message + internal data
• Logs state transitions
• Use with: akka.actor.debug.lifecycle=true
31

32. Debug log example
[DEBUG] [akka://fsm/user/receptionist] processing
Event(Result(Set([…])),Queue(1,Vector(Job(Actor[akka://fsm/system/testActor1#758674372],http://doc.akka.io
/docs/akka/2.3.5/intro/what-is-akka.html,1)))) from Actor[akka://fsm/user/receptionist/controller-1#4232237]
[DEBUG] [akka://fsm/user/receptionist/controller-1] transition CollectingResults -> Completed
[DEBUG] [akka://fsm/user/receptionist] transition Processing -> Sleeping
[DEBUG] [akka://fsm/user/receptionist/controller-1] stopped
32

33. [ERROR] [akka://fsm/user/receptionist] Unhandled event: some string
Last 8 entries leading up to this point:
in state: Sleeping
with data: NoQueue(2)
received: Scrape(http://non-existent.link,5)
in state: Processing
with data: Queue(3,Vector(Job(Actor[akka://fsm/system/testActor1#758674372],http://non-existent.
link,5)))
received: Result(Set(http://non-existent.link))
[…]
in state: Processing
with data:
Queue(4,Vector(Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent1,0),
Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent2,0),
Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent3,0),
Job(Actor[akka://fsm/system/testActor1#758674372],http://non.existent4,0)))
received: some string 33

34. Hell status
var ✔
No more mutable global state
inside Actors. Everything is
typed to the specific State.
become/unbecome ✔
All methods have to end in a
state transition. States are
clearly defined what they do.
debug/logging ✔
FSM does all the debug logging
we would ever need.
34

35. Conclusion: My experiences
Some people, when confronted with a scalability problem, think
“I know, I'll use Akka.” Now they have two problems.
• Akka is awesome, but:
• Needs a new mindset
• Unexperienced developers + learning curve + easy to make
mistakes = headaches
• In large teams (30+) where everyone is expected to be able to
change all code it becomes an issue.
• Use Actors only when really needed
• All Actors should probably be FSM
@krivachy https://github.com/krivachy/AkkaWithFsm