Developing distributed applications with Akka and Akka Cluster

1. Developing distributed applications
with Akka and Akka Cluster
Presented by K.Tsykulenko

2. Introduction

3. Agenda
• What is Akka?
• Concurrency paradigms overview.
• Actors and actor model.
• Live demo #1.
• Akka remoting and clustering.
• Live demo #2.
• CRDTs and Akka Distributed Data.
• Live demo #3.
• Summary.
• Q&A.

4. What is Akka?
http://akka.io
Akka is a toolkit and runtime for building highly
concurrent, distributed, and fault tolerant event-driven
applications on the JVM.

5. What is Akka?
http://akka.io

6. Concurrency paradigms

7. Concurrency paradigms
• Shared state and locks
• Software Transactional Memory (STM)
• Message-Passing Concurrency (Actors)
• Dataflow Concurrency
• and more…

8. Software transactional memory
V 1 V 2
V 1 V 12
V 11
V 3
V 2 V 22
successful
write
transaction
retried
write
transaction
V 1
Time

9. Dataflow Concurrency
X 1
X12=f(X1,X2)
X123=f(X12,X3)
X 2
X 3

10. Actors

11. Actors
• Originate in a 1973 paper by Carl Hewitt
• Implemented in Erlang
• Encapsulate state and behavior
• Closer to the definition of OO than classes

12. Actors
Sender ActorRef Actor
Dispatcher
hasMessageMessage
Message
Mailbox
has
enqueue dequeue
schedule
send

13. Actors
user ! User(“John Doe")
class UserActor extends Actor {
def receive = {
case User(name) => sender ! s"Hi $name"
}
}

14. Actors
val greeting = user ? User(“John Doe")
class UserActor extends Actor {
def receive = {
case User(name) => sender ! s"Hi $name"
}
}

15. Supervision and hierarchy
worker 1 worker 2 worker 3 worker 4
supervisor 1 supervisor 2
user

16. Building a web crawler
1. Fetch a page
2. Parse the page to get links
3. Check if max crawl depth has been reached and if
yes, finish
4. Go to 1 for all parsed links

17. Building a web crawler
Parser
CrawlMasteruser Fetcherpass urls
pass page content
pass parsed urls

18. Building a web crawler
Router
pass urls
CrawlMasterinitial url UrlHandlercreate UrlHandler
Fetcher Fetcher
Router
Parser Parser

19. Building a web crawler
class FetcherActor(val parser: ActorRef) extends Actor with ActorLogging {
import context.dispatcher
val pipeline: HttpRequest => Future[HttpResponse] = sendReceive
override def receive: Receive = {
case Url(link, depth) => pipeline(Get(link)).map(…).pipeTo(parser)
}
}

20. Building a web crawler
class ParserActor extends Actor with ActorLogging with HtmlParser {
override def receive: Receive = {
case UrlContents(Url(link, depth), resp, requester) =>
val links = parseHtml(resp)
.map(l => if (l.matches("^[/#].*")) link + l else l)
.filter(l => Try(new Url(l)).isSuccess
links.foreach(requester ! Url(_, depth + 1))
}
}

21. Building a web crawler
akka {
actor.deployment {
/parsers {
router = round-robin-pool nr-of-instances = 5
}
/fetchers {
router = round-robin-pool nr-of-instances = 5
}
}
}

22. Live demo #1

23. Going remote
• Everything works using asynchronous message
passing which is good for remoting
• Akka-remoting allows working with remote actors
just as if they were in the same JVM
• Still need to handle additional issues like
serialization and handling potential networking
problems

24. Akka cluster – pool routing
akka.tcp://localhost:2551/user
/crawler
Routees
akka.tcp://localhost:2552/user
/crawler
Cluster Pool
Router
localhost:2550
5. Gossip:
localhost:2551
is Up
2. Gossip:
localhost:2552
is Up
6. Deploy
routee
3. Deploy
routee
localhost:2551
localhost:2552
1. Joins cluster
4. Joins cluster
/remote/…/
(routee)
/remote/…/
(routee)

25. 1. Joins cluster
Akka cluster – group routing
akka.tcp://localhost:2551/user
/crawler
Routees
akka.tcp://localhost:2552/user
/crawler
Cluster Group
Router
localhost:2550
5. Gossip:
localhost:2551
is Up
2. Gossip:
localhost:2552
is Up
6. Routes
messages
3. Routes
messages
/user/crawler
localhost:2551
4. Joins cluster
/user/crawler
localhost:2552

26. Simple crawler cluster
Client VM
CrawlClient
Router
Worker VM
CrawlMaster(s)
Worker VM
CrawlMaster(s)

27. Going remote
cluster {
seed-nodes = [
"akka.tcp://CrawlerSystem@127.0.0.1:2551",
"akka.tcp://CrawlerSystem@127.0.0.1:2552"]
auto-down-unreachable-after = 10s
role {
client.min-nr-of-members = 1
backend.min-nr-of-members = 2
}
}

28. Going remote
actor {
deployment {
/workerRouter {
router = consistent-hashing-group
nr-of-instances = 100
routees.paths = ["/user/master"]
cluster {
enabled = on
allow-local-routees = on
use-role = backend
}
}
}
provider = "akka.cluster.ClusterActorRefProvider“
}

29. Live demo #2

30. Distributed state

31. Distributed state

32. But what if we need to?

33. CRDTs
• Good performance and scalability, the cost is
eventual consistency
• Two main classes: operation based and state based

34. CmRDTs

35. CvRDTs

36. Live demo #3

37. Summary
• Actor model provides a concurrency paradigm that
is easier to reason about than traditional Java
concurrency
• Designing actor systems is a lot like OO
• You can easily make your actor systems distributed
and have referential transparency… to an extent
• Akka has many useful modules, like Akka
distributed data, which allows to manage
distributed state
• Try to build your own application and see how it
works for you

38. References
• http://akka.io/
• http://www.allthingsdistributed.com/files/amazon-
dynamo-sosp2007.pdf
• https://vimeo.com/43903960

39. Q&A

40. Thank you!
Konstantin Tsykulenko
@Tsykulenko_K