JDD2015: Sharding with Akka Cluster: From Theory to Production - Krzysztof Otrebski, Michał Płachta

•

0 likes•130 views

SHARDING WITH AKKA CLUSTER: FROM THEORY TO PRODUCTION Writing distributed applications is very hard, especially when you start developing them as single-noded ones. Programmers tend to focus on functionalities first, leaving the scalability issues for later. Fortunately, Akka gives us many tools for scaling out and we can use them very early in the development process. You will learn how to take advantage of these features. The talk is divided into two sections. In the first section you will learn how to transform a single-noded app into a scalable one. During live coding session we will create both versions from scratch and guide you through the most important architectural decisions. Then we will move on to more important problems. During the second section we would like to simulate missile defence system and protect planet Earth against hostile aliens. Under the hood there will be Scala and Akka Cluster with sharding extension. After describing the application and its internals, we will simulate network and power failures to provide some idea how fast the application can recover.

Sharding with Akka Cluster
From Theory to Production
Michał Płachta
@miciek
Krzysztof Otrębski
@kotrebski

What’s inside?
● Creating web service using actor model
● ...making it scalable
● Playing with more complex app
● ...validating its fault tolerance

Akka Tutorial
● actor ~= thread
● actorRef.tell
● actorRef.ask
● actors create children
ActorRef
Sender 1 Sender 2
ask tell
enqueue
MailboxActor
dequeue

Scala Tutorial
● case class
● pattern matching
case class Junction(id: Int)
public class Junction {
private final int id;
public Junction(int id) {
this.id = id;
}
public int getId() {
return id;
}
// hashCode
// equals
// copy
}
msg match {
case Junction(id) =>
// this will execute
// when msg is Junction
case SomeOtherType =>
}

First example: Sorter
scan <containerId> -> HTTP -> push right or not
http://i.imgur.com/mctb4HC.gifv

Sorter Web Service
http://localhost:8080/junctions/<junctionId>/decisionForContainer/<containerId>
returns JSON
{ “direction”: left | right | straight | ... }
Assumptions:
● 5-10 ms to make a decision
● business logic defined - focus on performance

Step 1: Just REST...
RestInterface
HTTP Requests HTTP Responses
● One Actor = One Thread
● Blocking inside receive method
● Low throughput...

Throughput testing
/junctions/1/decisionForContainer/1
/junctions/2/decisionForContainer/4
/junctions/3/decisionForContainer/5
/junctions/4/decisionForContainer/2
/junctions/5/decisionForContainer/7
2000 requests
2000 requests
2000 requests
2000 requests
2000 requests
in parallel
cat URLs.txt | parallel -j 5 'ab -ql -n 2000 -c 1 -k {}'
GNU Parallel ApacheBench

Step 1: Just REST...
RestInterface
HTTP Requests HTTP Responses
± % cat URLs.txt | parallel -j 5 'ab -ql -n 2000 -c 1 -k {}' | grep 'Requests per second'
Requests per second: 34.78 [#/sec] (mean)
Requests per second: 34.22 [#/sec] (mean)
Requests per second: 33.77 [#/sec] (mean)
Requests per second: 33.82 [#/sec] (mean)
Requests per second: 33.98 [#/sec] (mean)

Step 1.5: Logic in another actor
RestInterface
HTTP Requests HTTP Responses
SortingDecider

Step 2: One actor per junction
RestInterface
HTTP Requests HTTP Responses
DecidersGuardian
SortingDecider
SortingDecider
SortingDecider
<junctionId>=1 ... <junctionId>=5

Step 2: One actor per junction
± % cat URLs.txt | parallel -j 5 'ab -ql -n 2000 -c 1 -k {}' | grep 'Requests per second'
Requests per second: 67.36 [#/sec] (mean)
Requests per second: 69.03 [#/sec] (mean)
Requests per second: 67.75 [#/sec] (mean)
Requests per second: 66.88 [#/sec] (mean)
Requests per second: 66.28 [#/sec] (mean)

Now what?
● non-blocking
● concurrent
● scaling up works
● scaling out?
RestInterface
HTTP Requests HTTP Responses
DecidersGuardian
SortingDecider
SortingDecider
SortingDecider
<junctionId>=1 ... <junctionId>=5

Manual scaling out
RestInterface
HTTP Requests HTTP Responses
DecidersGuardian
SortingDecider
SortingDecider
SortingDecider
<junctionId>=1 ... <junctionId>=3
RestInterface
HTTP Requests HTTP Responses
DecidersGuardian
SortingDecider
SortingDecider
SortingDecider
<junctionId>=4 ... <junctionId>=6

Enter Sharding
RestInterface
HTTP Requests HTTP Responses
ShardRegion
SortingDecider
SortingDecider
SortingDecider
<junctionId>=h(m) ... <junctionId>=h(m)
RestInterface
HTTP Requests HTTP Responses
ShardRegion
SortingDecider
SortingDecider
SortingDecider
<junctionId>=h(m) ... <junctionId>=h(m)
...

Step 3: Sharded web service
± % cat URLs.txt | parallel -j 5 'ab -ql -n 2000 -c 1 -k {}' | grep 'Requests per second'
Requests per second: 106.80 [#/sec] (mean)
Requests per second: 108.15 [#/sec] (mean)
Requests per second: 100.60 [#/sec] (mean)
Requests per second: 99.92 [#/sec] (mean)
Requests per second: 100.07 [#/sec] (mean)

Sharding
● automatic distribution
● no need to know who is where
● no need to know how many nodes are there
● rebalancing
● migration

https://it.wikipedia.org/wiki/File:Richard-feynman.jpg

...it seems to have been the general policy then to
not try to repeat psychological experiments,
Richard Feynman

Some important cluster settings
akka {
cluster {
auto-down-unreachable-after = 20s
}
}

What you know now
How Akka Cluster and Sharding works
It's good to run failover tests before going on protoduction
Close your nodes gracefully

How can I run those apps?
● https://github.com/miciek/akka-sharding-example
● https://github.com/otrebski/reactive-missile-defend

Thank you!
Any questions?
Michał Płachta
@miciek
Krzysztof Otrębski
@kotrebski

Concurrency control is mostly based on locks and is therefore notoriously difficult to use. Even though some programming languages provide high-level constructs, these add complexity and potentially hard-to-detect bugs to the application. Transactional memory is an attractive mechanism that does not have the drawbacks of locks, however the underlying implementation is often difficult to integrate into an existing language. In this paper we show how we have introduced transactional semantics into Smalltalk by using the reflective facilities of the language. Our approach is based on method annotations, incremental parse tree transformations and an optimistic commit protocol. The implementation does not depend on modifications to the virtual machine and therefor can be changed at the language level. We report on a practical case study, benchmarks and further and on-going work.

G pars

NexThoughts Technologies

GPars (Groovy Parallel Systems) is an open-source concurrency and parallelism library for Java and Groovy that gives you a number of high-level abstractions for writing concurrent and parallel code in Groovy (map/reduce, fork/join, asynchronous closures, actors, agents, dataflow concurrency and other concepts), which can make your Java and Groovy code concurrent and/or parallel with little effort.

Testing in android

jtrindade

This talk, presented at I.T.A.K.E 2013, explores options for revising, repairing, and extending good, bad, and ugly code. It takes a hard look at some good, bad, and ugly code written by the presenter and others. Most people work with code that has both good and bad parts. The challenge with such a mix is how to modify what’s there without damaging the good parts, making bad things worse, or ugly situations even uglier. Sometimes we get an opportunity to make major repairs. But more often, we can only make small, incremental improvements. What we can do is constrained by cost, consistency, and compatibility concerns. Living with good, bad, and even ugly code is never easy. But as Clint Eastwood says, “Sometimes if you want to see a change for the better, you have to take things into your own hands.”

Scaling django

Md. Ahsanuzzaman Khan

Writing robust Node.js applicationsTom Croucher

Testing RESTful Webservices using the REST-assured framework

Micha Kops

Multithreading and concurrency in android

Rakesh Jha

CLS & asyncListener: asynchronous observability for Node.js

Forrest Norvell

Async JavaScript Unit Testing

Mihail Gaberov

Introduction to nodejsJames Carr

Javascript Promises/Q Library

async_io

NodeJS for Beginner

Apaichon Punopas

NodeJS.toster

5 Steps to Mastering the Art of Seaside

Lukas Renggli

Seaside does things differently from what is considered best practice for Web development. Seaside breaks with common best practices, such as sharing as little state as possible, using clean and carefully chosen URLs, and using templates to separate model and presentation. This tutorial will give a quick introduction to a Web framework that is different by design. It will demonstrate new patterns of Web development, that let you build highly interactive Web applications quickly, reusably and maintainably. Moreover it will show how Seaside integrates with latest technologies such as AJAX and Comet. Lukas Renggli is a core developer of the Seaside web application framework. He has been using Seaside in industrial settings for more than 5 years. Lukas Renggli is the author of several frameworks built on top of Seaside, such as the Pier Content Management System.

Top 30 Node.js interview questions

techievarsity

Avoiding Callback Hell with Async.js

cacois

Efficient Android ThreadingAnders Göransson

Automatic Reference Counting

Robert Brown

Intro to Asynchronous Javascript

Garrett Welson

Atmosphere Conference 2015: Bottoms-up and back DevOps

PROIDEA

Speaker: Pawel Bartusch Language: English I’m going to tell you a story. A story about a small group of people that dared to challenge large enterprise status quo. I’ll tell you how we’ve learned Chef, Git and Jenkins and how we use it to build products of growing complexity. I’m going to reveal our recipe on how to hunt an opportunity and turn it into credibility. I’ll tell you how we’ve started experimenting with scrum, how we’ve learned to use it effectively and finally how we scaled it across a team of 50 engineers working on a multimillion dollar project. The best thing about the story is that it’s nowhere near finished. The final outcome is yet be seen… Visit our website: http://atmosphere-conference.com/

CONFidence 2015: Let's play SOCer Security Operations Center i Teoria Chaosu ...

PROIDEA

What's hot

Going towards inversion of control and better designoazabir

arataga. SObjectizer and RESTinio in action: a real-world example

Yauheni Akhotnikau

Guice gin

Robert Cooper

Maintaining Your Code Clint Eastwood Style

Rebecca Wirfs-Brock

Scaling django

Md. Ahsanuzzaman Khan

Writing robust Node.js applicationsTom Croucher

Testing RESTful Webservices using the REST-assured framework

Micha Kops

Multithreading and concurrency in android

Rakesh Jha

CLS & asyncListener: asynchronous observability for Node.js

Forrest Norvell

Async JavaScript Unit Testing

Mihail Gaberov

Introduction to nodejsJames Carr

Javascript Promises/Q Library

async_io

NodeJS for Beginner

Apaichon Punopas

NodeJS.toster

5 Steps to Mastering the Art of Seaside

Lukas Renggli

Top 30 Node.js interview questions

techievarsity

Avoiding Callback Hell with Async.js

cacois

Efficient Android ThreadingAnders Göransson

Automatic Reference Counting

Robert Brown

Intro to Asynchronous Javascript

Garrett Welson

What's hot (20)

Going towards inversion of control and better design

arataga. SObjectizer and RESTinio in action: a real-world example

Guice gin

Maintaining Your Code Clint Eastwood Style

Scaling django

Writing robust Node.js applications

Testing RESTful Webservices using the REST-assured framework

Multithreading and concurrency in android

CLS & asyncListener: asynchronous observability for Node.js

Async JavaScript Unit Testing

Introduction to nodejs

Javascript Promises/Q Library

NodeJS for Beginner

NodeJS

5 Steps to Mastering the Art of Seaside

Top 30 Node.js interview questions

Avoiding Callback Hell with Async.js

Efficient Android Threading

Automatic Reference Counting

Intro to Asynchronous Javascript

Viewers also liked

Atmosphere Conference 2015: Bottoms-up and back DevOps

PROIDEA

CONFidence 2015: Let's play SOCer Security Operations Center i Teoria Chaosu ...

PROIDEA

CONFidence 2015: APT x 3 - trzy firmy, trzy wektory ataków, trzy do zera - wy...

PROIDEA

Speaker: Borys Lacki Language: Polish Historia trzech ataków typu APT wykonanych podczas kontrolowanych testów penetracyjnych. W których od zerowej wiedzy na temat atakowanej firmy przejęto infrastrukturę i uzyskano dostęp do poufnych informacji wykorzystując 0-day w Quake, pendrive w toalecie damskiej oraz plik faktura.exe. Oprócz mrożących krew w żyłach historii zostaną pokazane praktyczne porady jak nie stać się bohaterem kolejnej edycji wykładu. CONFidence: http://confidence.org.pl/pl/

4Developers 2015: Couple of words about testing in Java, Spock and BDD - Piot...

PROIDEA

Speaker: Piotr Kiebasiński Language: Polish Testy jednostkowe traktowane są często przez programistów jako zło konieczne, coś co powoduje opóźnienie przy dostarczeniu oprogramowania albo niepotrzebnie dokłada pracy. Podczas wykładu będę chciał pokazać, że testowanie może mieć sens, być efektywne, zmniejszyć ilość czasu spędzonego nad kodem i wydatnie podnieść zarówno tempo pracy jak i jej jakość. Odwołam się przy tym do metodyki BDD oraz do wykorzystania frameworka Spock. 4Developers: http://4developers.org.pl/pl/

Atmosphere Conference 2015: Oktawave Horizon Project: the future of real-time...

PROIDEA

Atmosphere Conference 2015: Do you think you're doing microservices?

PROIDEA

Speakers: Marcin Grzejszczak, Łukasz Szczesny Language: English Microservices are the top buzzword in IT recently. Rarely people think that it’s not just about having a separate codebase for each service. Even if you produce that fat jar from your code - what happens now? How does your deployment pipeline look like? Having a microservice based architecure implies heavy impact on the infrastructure and its automatic provisioning. Distributed systems need to have their logs and metrics aggregated. No more fixed ports and addresses definitions. Do you actually know what service discovery means? If you haven’t thought about this then you shouldn’t go the microservice way. We will explain what microservices are, why they are not that trivial to deal with and how to automatically set up the infrastructure around them. Visit our website: http://atmosphere-conference.com/

PLNOG15: Internet of things and modern M2M solutions. New market for new serv...

PROIDEA

Atmosphere Conference 2015: Building And Releasing A Massively Multiplayer On...

PROIDEA

Speaker: Jamie Winsor Language: English When a small startup of experienced game developers in Seattle finished their first game, an open-world single player zombie survival game, to rave reviews they continuously received the same feedback - "No multiplayer?". Building the necessary software infrastructure for a massively multiplayer online game historically requires large engineering teams, even larger operational support organizations, and can take years to finish. With a big success under their belts the organization had a choice to make: Go through a massive growth period, potentially ruining the culture they worked so hard to cultivate or hire smart, work even smarter, and use the right tools for the job. In this talk you will experience this story from to back. Come hear how Undead Labs grew from a small 20 person game development shop creating a single player Xbox 360 Live Arcade Game, State of Decay, to a multi-game studio bringing State of Decay and more to the online space. You will hear how Undead Lab's leveraged Chef, Elixir, and fully embraced the DevOps development pattern to create the foundations of their new online platform while simultaneously developing multiple titles while cultivating the personnel required to lock in a bright future of the studio. Visit our website: http://atmosphere-conference.com/

PLNOG15: NFV: Lessons learned from production deployments and current observa...

PROIDEA

CONFidence 2015: Fuzz your way into the web server's zoo - Andrey Plastunov

PROIDEA

Speaker: Andrey Plastunov Language: English Nowadays, one can find many many different web servers performing different kind of tasks: they may serve administrative requests on wifi hotspots (or any other embedded device), they may be used as gates to corporate intranet resources, etc. That's the logic part, but what do all these servers have under their hoods? Attentive people may note that the zoo of web servers is not limited to popular mainstream solutions. So how large the zoo is exactly? Do the developers, who creates their own servers, pay enough attention to security related problems? We will try to answer that questions. The talk will cover methods of finding binary vulnerabilities in modern web server software using A fuzzing approach. As an example of this method, a custom fuzzing tool will be released. We will also demonstrate a bunch of vulnerabilities, found during the research. CONFidence: http://confidence.org.pl/pl/

PLNOG15 - Wi-Fi Calling – how any Wi-FI infrastructure can become a part of M...

PROIDEA

Atmosphere Conference 2015: PubSub++ - few tips that make your life with kafk...

PROIDEA

Speaker: Krzysztof Debski Language: English You’ve just set up your Kafka cluster and now you are ready to process tens of thousands events per second. You decoupled your architecture and now all the communication goes via pubSub bus and you can focus only on providing business value. It would be great if that can be true. In real life you need to do a lot of tweaks to have your backbone ready to handle all the traffic you want. In the talk we will focus on the project of redesigning the core event bus of the Allegro auction platform. We will talk issues on all the levels from network to data serialization. After the presentation you will have answers to the questions: How to assure our clients that not even a single event is lost? How to respond in less than 65ms even if Kafka is rebuilding its cluster? ...And many other questions you will face. Visit our website: http://atmosphere-conference.com/

PLNOG15: How to effectively build the networks with 1.1 POPC programme? - Mar...

PROIDEA

CONFidence 2015: iOS Hacking: Advanced Pentest & Forensic Techniques - Omer S...

PROIDEA

Speaker: Omer S. Coskun Language: English While there has certainly been valuable interesting research of blackbox security assessments techniques presented on different conferences, it exclusively has almost focused on application layer of iOS. The recent disclosures on surveillance programs suggests that mobile users also being targeted not only by cyber criminals but also spy agencies. The level of skill and effort to prevent such an attack requires a reproducible threat model - a REDteam exercise. This talk appeals to hands-on iOS hackers looking to dive into iOS Security Architecture, Sandbox mechanism, ARM64 assembly and Security APIs while being firmly accompanied with always overlooked penetration testing techniques and the ways of how to automate them. The talk will cover dynamic memory reversing and how to tackle cryptography on an assessment so that participants will understand how to quantitatively and qualitatively carry an offensive penetration testing or forensic examination of iOS environment. CONFidence: http://confidence.org.pl/pl/

CONFidence 2015: Defensive Time-Out or unclear digressions about past present...

PROIDEA

Speaker: Blazej Boczula Language: English This presentation is not about computer hacking, penetration testing or live demonstration of security flaws. It’s not even technical in strict sense and due to the large span of discussed topics hard to estimate the worth, merit or value of it. I sn’t this just a waste of time? M aybe. But to answer this question shouldn’t we first stop and recall: What actually is ‘time’? The main purpose of my talk is to identify problems and pose some questions about time observed from Security perspective with particular reference to CyberSecurity. Let’s imagine that we have all the time in the world to hear about concepts and ideas of Mills, Essen, Lamport, Anderson, Einstein, Ajdukiewicz, Dali, Eddington, Augustynek, Whitrow, Wells, Luhmann to mention only a few. Kill some time puzzling about t ime zones, time protocols, time correlation, time perception, time measurement, uptime, timing attacks, realtime, total exhaust time,time stamps, epoch time, expiration time, time series, time travel, time loop, boot time, time paradox, polynomial time, exponential time, timelapse cryptography, working time, time off, physical time, timesharing, time limit, time bomb, time control, time standard, time signal, lamport time, time interval, system time, time complexity, time forwarding and storage bugs, time to live, time machine, time code ambiguity, time capsule, time function, time dilation, time expressions, time book, timesheet, timekeeper, time management, time warping, time servers, units of time, time periods, bullet time, proper time, fixed time, overtime, lifetime and more. ALLERGY ADVICE: T his presentation may contain philosophy, art, humanities and science, which are used during the manufacturing process. CONFidence: http://confidence.org.pl/pl/

Power and politics

Gc university faisalabad

PLNOG15: Data Center migration in practice - Tomasz Jarlaczyk

PROIDEA

PLNOG15: Network Automation and Programmability Abstraction Layer with Multiv...

PROIDEA

PLNOG15: Network Migration and Service Assurance using Smart SFP Modules - To...

PROIDEA

4Developers 2015: Sprytniejsze testowanie kodu Java ze Spock Framework - Marc...

PROIDEA

Marcin Zajączkowski Language: English Spock Framework jest coraz częściej wykorzystywany do pisania testów automatycznych dla kodu produkcyjnego napisanego w Javie. Dzięki zastosowaniu DSLa opartego o język Groovy umożliwia tworzenie bardziej zwartych konstrukcji charakteryzujących się dużą czytelnością. W czasie prezentacji pokażę szereg bardziej zaawansowanych i często mało znanych mechanizmów, których zastosowanie może jeszcze bardziej ułatwić i uprościć testowanie naszego kodu. Przedstawię również kilka nowych funkcji, które stały się dostępne wraz z wydaniem jeszcze ciepłej wersji 1.0. O podstawach Spocka mówić nie będę, ale dzięki jego naturalnej klarowności nawet osoby wcześniej go nieużywające na pewno nie będą miały trudności, aby się odnaleźć.

Viewers also liked (20)