SlideShare a Scribd company logo
1 of 52
Download to read offline
akka streams
Reactive Integrations with
that just work™
Johan Andrén
Konrad Malawski
Johan Andrén
Akka Team
Stockholm Scala User Group
Konrad `ktoso` Malawski
Akka Team,
Reactive Streams TCK,
Persistence, HTTP
Make building powerful concurrent &
distributed applications simple.
Akka is a toolkit and runtime
for building highly concurrent,
distributed, and resilient
message-driven applications
on the JVM
Actors – simple & high performance concurrency
Cluster / Remoting – location transparency, resilience
Cluster tools – and more prepackaged patterns
Streams – back-pressured stream processing
Persistence – Event Sourcing
HTTP – complete, fully async and reactive HTTP Server
Official Kafka, Cassandra, DynamoDB integrations, tons
more in the community
Complete Java & Scala APIs for all features
What’s in the toolkit?
“Stream”
has many meanings
akka streams
Asynchronous back pressured stream processing
Source Sink
Flow
akka streams
Asynchronous back pressured stream processing
Source Sink
(possible)
asynchronous
boundaries
Flow
akka streams
Asynchronous back pressured stream processing
Source Sink
10 msg/s 1 msg/s
OutOfMemoryError!!
Flow
akka streams
Asynchronous back pressured stream processing
Source Sink
10 msg/s 1 msg/s
hand me 3 morehand me 3 more
1 msg/s Flow
akka streams
Not only linear streams
Source
SinkFlow
Source
Sink
Flow
Flow
Reactive Streams
Reactive Streams is an initiative to provide a standard for
asynchronous stream processing with non-blocking back
pressure. This encompasses efforts aimed at runtime
environments as well as network protocols
http://www.reactive-streams.org
Part of JDK 9
java.util.concurrent.Flow
Reactive Streams
RS Library A RS library B
async
boundary
Reactive Streams
RS Library A RS library B
async
boundary
Make building powerful concurrent &
distributed applications simple.
The API
Akka Streams
Complete and awesome
Java and Scala APIs
(Just like everything in Akka)
Akka Streams in 20 seconds:
Source<Integer, NotUsed> source = null;



Flow<Integer, String, NotUsed> flow =

Flow.<Integer>create().map((Integer n) -> n.toString());



Sink<String, CompletionStage<Done>> sink =

Sink.foreach(str -> System.out.println(str));



RunnableGraph<NotUsed> runnable =
source.via(flow).to(sink);



runnable.run(materializer);

Akka Streams in 20 seconds:
CompletionStage<String> firstString =

Source.single(1)

.map(n -> n.toString())

.runWith(Sink.head(), materializer);

Source.single(1).map(i -> i.toString).runWith(Sink.head())
// types: _
Source<Int, NotUsed>
Flow<Int, String, NotUsed>
Sink<String, CompletionStage<String>>
Akka Streams in 20 seconds:
Source.single(1).map(i -> i.toString).runWith(Sink.head())
// types: _
Source<Int, NotUsed>
Flow<Int, String, NotUsed>
Sink<String, CompletionStage<String>>
Akka Streams in 20 seconds:
Materialization
Gears from GeeCON.org,(it’s an awesome conf)
What is “materialization” really?
What is “materialization” really?
What is “materialization” really?
What is “materialization” really?
What is “materialization” really?
Check out the
“Implementing an akka-streams materializer for big data”
talk later today.
AlpakkaA community for Streams connectors
http://blog.akka.io/integrations/2016/08/23/intro-alpakka
Alpakka – a community for Stream connectors
Threading & Concurrency in Akka Streams Explained (part I)
Mastering GraphStages (part I, Introduction)
Akka Streams Integration, codename Alpakka
A gentle introduction to building Sinks and Sources using GraphStage APIs
(Mastering GraphStages, Part II)
Writing Akka Streams Connectors for existing APIs
Flow control at the boundary of Akka Streams and a data provider
Akka Streams Kafka 0.11
Alpakka – a community for Stream connectors
Existing examples:
MQTT
AMQP
Streaming HTTP
Streaming TCP
Streaming FileIO
Cassandra Queries
“Reactive Kafka” (akka-stream-kafka)
S3, SQS & other Amazon APIs
Streaming JSON
Streaming XML
…
Alpakka – a community for Stream connectors
Demo
Alpakka – a community for Stream connectors
Demo
Akka Streams & HTTP
streams
& HTTP
A core feature not obvious to the untrained eye…!
Akka Streams / HTTP
Quiz time!
TCP is a ______ protocol?
A core feature not obvious to the untrained eye…!
Akka Streams / HTTP
Quiz time!
TCP is a STREAMING protocol!
Streaming in Akka HTTP
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
HttpServer as a:
Flow[HttpRequest, HttpResponse]
Streaming in Akka HTTP
HttpServer as a:
Flow[HttpRequest, HttpResponse]
HTTP Entity as a:
Source[ByteString, _]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
Streaming in Akka HTTP
HttpServer as a:
Flow[HttpRequest, HttpResponse]
HTTP Entity as a:
Source[ByteString, _]
Websocket connection as a:
Flow[ws.Message, ws.Message]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
It’s turtles buffers all the way down!
xkcd.com
Streaming from Akka HTTP
Streaming from Akka HTTP
Streaming from Akka HTTP
No demand from TCP
=
No demand upstream
=
Source won’t generate tweets
=>
Bounded memory
stream processing!
Demo
Streaming from Akka HTTP (Java)
public static void main(String[] args) {
final ActorSystem system = ActorSystem.create();
final Materializer materializer = ActorMaterializer.create(system);
final Http http = Http.get(system);
final Source<Tweet, NotUsed> tweets = Source.repeat(new Tweet("Hello world"));
final Route tweetsRoute =
path("tweets", () ->
completeWithSource(tweets, Jackson.marshaller(), EntityStreamingSupport.json())
);
final Flow<HttpRequest, HttpResponse, NotUsed> handler =
tweetsRoute.flow(system, materializer);
http.bindAndHandle(handler,
ConnectHttp.toHost("localhost", 8080),
materializer
);
System.out.println("Running at http://localhost:8080");
}
Streaming from Akka HTTP (Java)
public static void main(String[] args) {
final ActorSystem system = ActorSystem.create();
final Materializer materializer = ActorMaterializer.create(system);
final Http http = Http.get(system);
final Source<Tweet, NotUsed> tweets = Source.repeat(new Tweet("Hello world"));
final Route tweetsRoute =
path("tweets", () ->
completeWithSource(tweets, Jackson.marshaller(), EntityStreamingSupport.json())
);
final Flow<HttpRequest, HttpResponse, NotUsed> handler =
tweetsRoute.flow(system, materializer);
http.bindAndHandle(handler,
ConnectHttp.toHost("localhost", 8080),
materializer
);
System.out.println("Running at http://localhost:8080");
}
Streaming from Akka HTTP (Scala)
object Example extends App
with SprayJsonSupport with DefaultJsonProtocol {
import akka.http.scaladsl.server.Directives._
implicit val system = ActorSystem()
implicit val mat = ActorMaterializer()
implicit val jsonRenderingMode = EntityStreamingSupport.json()
implicit val TweetFormat = jsonFormat1(Tweet)
def tweetsStreamRoutes =
path("tweets") {
complete {
Source.repeat(Tweet(""))
}
}
Http().bindAndHandle(tweetsStreamRoutes, "127.0.0.1", 8080)
System.out.println("Running at http://localhost:8080");
}
Next steps for Akka
Completely new Akka Remoting (goal: 700.000+ msg/s (!)),
(it is built using Akka Streams, Aeron).
More integrations for Akka Streams stages, project Alpakka.
Reactive Kafka polishing with SoftwareMill, Krzysiek Ciesielski
Akka Typed progressing again, likely towards 3.0.
Akka HTTP 2.0 Proof of Concept in progress.
Collaboration with Reactive Sockets
Ready to adopt on prod?
Totally, go for it.
Akka <3 contributions
Easy to contribute tickets:
https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute
https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22
Akka Stream Contrib
https://github.com/akka/akka-stream-contrib
Mailing list:
https://groups.google.com/group/akka-user
Public chat rooms:
http://gitter.im/akka/dev developing Akka
http://gitter.im/akka/akka using Akka
Reactive Platform
Reactive Platform
Reactive Platform
Further reading:
Reactive Streams: reactive-streams.org
Akka documentation: akka.io/docs
Free O’Reilly report – very out soon.
Example Sources:
ktoso/akka-streams-alpakka-talk-demos-2016
Get involved:
sources: github.com/akka/akka
mailing list: akka-user @ google groups
gitter channel: https://gitter.im/akka/akka
Contact:
Konrad ktoso@lightbend.com Malawski
http://kto.so / @ktosopl
Thanks!
Questions?
@apnylle johan.andren@lightbend.com
@ktosopl konrad.malawski@lightbend.com

More Related Content

What's hot

Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka StreamsFresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Konrad Malawski
 
Reactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka StreamsReactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka Streams
Konrad Malawski
 
System Integration with Akka and Apache Camel
System Integration with Akka and Apache CamelSystem Integration with Akka and Apache Camel
System Integration with Akka and Apache Camel
krasserm
 

What's hot (20)

Not Only Streams for Akademia JLabs
Not Only Streams for Akademia JLabsNot Only Streams for Akademia JLabs
Not Only Streams for Akademia JLabs
 
The Need for Async @ ScalaWorld
The Need for Async @ ScalaWorldThe Need for Async @ ScalaWorld
The Need for Async @ ScalaWorld
 
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYCBuilding a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
 
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
 
2014 akka-streams-tokyo-japanese
2014 akka-streams-tokyo-japanese2014 akka-streams-tokyo-japanese
2014 akka-streams-tokyo-japanese
 
Networks and Types - the Future of Akka @ ScalaDays NYC 2018
Networks and Types - the Future of Akka @ ScalaDays NYC 2018Networks and Types - the Future of Akka @ ScalaDays NYC 2018
Networks and Types - the Future of Akka @ ScalaDays NYC 2018
 
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka StreamsFresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
 
Need for Async: Hot pursuit for scalable applications
Need for Async: Hot pursuit for scalable applicationsNeed for Async: Hot pursuit for scalable applications
Need for Async: Hot pursuit for scalable applications
 
Reactive Streams: Handling Data-Flow the Reactive Way
Reactive Streams: Handling Data-Flow the Reactive WayReactive Streams: Handling Data-Flow the Reactive Way
Reactive Streams: Handling Data-Flow the Reactive Way
 
Reactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka StreamsReactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka Streams
 
Reactive Streams / Akka Streams - GeeCON Prague 2014
Reactive Streams / Akka Streams - GeeCON Prague 2014Reactive Streams / Akka Streams - GeeCON Prague 2014
Reactive Streams / Akka Streams - GeeCON Prague 2014
 
ScalaSwarm 2017 Keynote: Tough this be madness yet theres method in't
ScalaSwarm 2017 Keynote: Tough this be madness yet theres method in'tScalaSwarm 2017 Keynote: Tough this be madness yet theres method in't
ScalaSwarm 2017 Keynote: Tough this be madness yet theres method in't
 
The things we don't see – stories of Software, Scala and Akka
The things we don't see – stories of Software, Scala and AkkaThe things we don't see – stories of Software, Scala and Akka
The things we don't see – stories of Software, Scala and Akka
 
Building reactive distributed systems with Akka
Building reactive distributed systems with Akka Building reactive distributed systems with Akka
Building reactive distributed systems with Akka
 
Akka Actor presentation
Akka Actor presentationAkka Actor presentation
Akka Actor presentation
 
Journey into Reactive Streams and Akka Streams
Journey into Reactive Streams and Akka StreamsJourney into Reactive Streams and Akka Streams
Journey into Reactive Streams and Akka Streams
 
Streaming all the things with akka streams
Streaming all the things with akka streams   Streaming all the things with akka streams
Streaming all the things with akka streams
 
VJUG24 - Reactive Integrations with Akka Streams
VJUG24  - Reactive Integrations with Akka StreamsVJUG24  - Reactive Integrations with Akka Streams
VJUG24 - Reactive Integrations with Akka Streams
 
Reactive Web-Applications @ LambdaDays
Reactive Web-Applications @ LambdaDaysReactive Web-Applications @ LambdaDays
Reactive Web-Applications @ LambdaDays
 
System Integration with Akka and Apache Camel
System Integration with Akka and Apache CamelSystem Integration with Akka and Apache Camel
System Integration with Akka and Apache Camel
 

Viewers also liked

Exploring Reactive Integrations With Akka Streams, Alpakka And Apache Kafka
Exploring Reactive Integrations With Akka Streams, Alpakka And Apache KafkaExploring Reactive Integrations With Akka Streams, Alpakka And Apache Kafka
Exploring Reactive Integrations With Akka Streams, Alpakka And Apache Kafka
Lightbend
 
Building Streaming And Fast Data Applications With Spark, Mesos, Akka, Cassan...
Building Streaming And Fast Data Applications With Spark, Mesos, Akka, Cassan...Building Streaming And Fast Data Applications With Spark, Mesos, Akka, Cassan...
Building Streaming And Fast Data Applications With Spark, Mesos, Akka, Cassan...
Lightbend
 
Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And...
Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And...Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And...
Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And...
Lightbend
 

Viewers also liked (8)

What's The Role Of Machine Learning In Fast Data And Streaming Applications?
What's The Role Of Machine Learning In Fast Data And Streaming Applications?What's The Role Of Machine Learning In Fast Data And Streaming Applications?
What's The Role Of Machine Learning In Fast Data And Streaming Applications?
 
Akka streams kafka kinesis
Akka streams kafka kinesisAkka streams kafka kinesis
Akka streams kafka kinesis
 
Akka Streams - From Zero to Kafka
Akka Streams - From Zero to KafkaAkka Streams - From Zero to Kafka
Akka Streams - From Zero to Kafka
 
Moving from Big Data to Fast Data? Here's How To Pick The Right Streaming Engine
Moving from Big Data to Fast Data? Here's How To Pick The Right Streaming EngineMoving from Big Data to Fast Data? Here's How To Pick The Right Streaming Engine
Moving from Big Data to Fast Data? Here's How To Pick The Right Streaming Engine
 
Exploring Reactive Integrations With Akka Streams, Alpakka And Apache Kafka
Exploring Reactive Integrations With Akka Streams, Alpakka And Apache KafkaExploring Reactive Integrations With Akka Streams, Alpakka And Apache Kafka
Exploring Reactive Integrations With Akka Streams, Alpakka And Apache Kafka
 
Building Streaming And Fast Data Applications With Spark, Mesos, Akka, Cassan...
Building Streaming And Fast Data Applications With Spark, Mesos, Akka, Cassan...Building Streaming And Fast Data Applications With Spark, Mesos, Akka, Cassan...
Building Streaming And Fast Data Applications With Spark, Mesos, Akka, Cassan...
 
Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And...
Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And...Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And...
Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And...
 
Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...
Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...
Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...
 

Similar to Reactive integrations with Akka Streams

Streaming Design Patterns Using Alpakka Kafka Connector (Sean Glover, Lightbe...
Streaming Design Patterns Using Alpakka Kafka Connector (Sean Glover, Lightbe...Streaming Design Patterns Using Alpakka Kafka Connector (Sean Glover, Lightbe...
Streaming Design Patterns Using Alpakka Kafka Connector (Sean Glover, Lightbe...
confluent
 

Similar to Reactive integrations with Akka Streams (20)

Akka streams - Umeå java usergroup
Akka streams - Umeå java usergroupAkka streams - Umeå java usergroup
Akka streams - Umeå java usergroup
 
Reactive stream processing using Akka streams
Reactive stream processing using Akka streams Reactive stream processing using Akka streams
Reactive stream processing using Akka streams
 
Asynchronous stream processing with Akka Streams
Asynchronous stream processing with Akka StreamsAsynchronous stream processing with Akka Streams
Asynchronous stream processing with Akka Streams
 
Reactive streams processing using Akka Streams
Reactive streams processing using Akka StreamsReactive streams processing using Akka Streams
Reactive streams processing using Akka Streams
 
Scala usergroup stockholm - reactive integrations with akka streams
Scala usergroup stockholm - reactive integrations with akka streamsScala usergroup stockholm - reactive integrations with akka streams
Scala usergroup stockholm - reactive integrations with akka streams
 
Building Stateful Microservices With Akka
Building Stateful Microservices With AkkaBuilding Stateful Microservices With Akka
Building Stateful Microservices With Akka
 
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & Ka...
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & Ka...Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & Ka...
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & Ka...
 
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & Kafka
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & KafkaBack-Pressure in Action: Handling High-Burst Workloads with Akka Streams & Kafka
Back-Pressure in Action: Handling High-Burst Workloads with Akka Streams & Kafka
 
Reactive Streams 1.0 and Akka Streams
Reactive Streams 1.0 and Akka StreamsReactive Streams 1.0 and Akka Streams
Reactive Streams 1.0 and Akka Streams
 
Understanding Akka Streams, Back Pressure, and Asynchronous Architectures
Understanding Akka Streams, Back Pressure, and Asynchronous ArchitecturesUnderstanding Akka Streams, Back Pressure, and Asynchronous Architectures
Understanding Akka Streams, Back Pressure, and Asynchronous Architectures
 
Let the alpakka pull your stream
Let the alpakka pull your streamLet the alpakka pull your stream
Let the alpakka pull your stream
 
Akka A to Z: A Guide To The Industry’s Best Toolkit for Fast Data and Microse...
Akka A to Z: A Guide To The Industry’s Best Toolkit for Fast Data and Microse...Akka A to Z: A Guide To The Industry’s Best Toolkit for Fast Data and Microse...
Akka A to Z: A Guide To The Industry’s Best Toolkit for Fast Data and Microse...
 
Building scalable rest service using Akka HTTP
Building scalable rest service using Akka HTTPBuilding scalable rest service using Akka HTTP
Building scalable rest service using Akka HTTP
 
Server side JavaScript: going all the way
Server side JavaScript: going all the wayServer side JavaScript: going all the way
Server side JavaScript: going all the way
 
Akka Microservices Architecture And Design
Akka Microservices Architecture And DesignAkka Microservices Architecture And Design
Akka Microservices Architecture And Design
 
Introduction to apache kafka, confluent and why they matter
Introduction to apache kafka, confluent and why they matterIntroduction to apache kafka, confluent and why they matter
Introduction to apache kafka, confluent and why they matter
 
Streaming Design Patterns Using Alpakka Kafka Connector (Sean Glover, Lightbe...
Streaming Design Patterns Using Alpakka Kafka Connector (Sean Glover, Lightbe...Streaming Design Patterns Using Alpakka Kafka Connector (Sean Glover, Lightbe...
Streaming Design Patterns Using Alpakka Kafka Connector (Sean Glover, Lightbe...
 
JS everywhere 2011
JS everywhere 2011JS everywhere 2011
JS everywhere 2011
 
Introduction to Akka Streams [Part-I]
Introduction to Akka Streams [Part-I]Introduction to Akka Streams [Part-I]
Introduction to Akka Streams [Part-I]
 
Asynchronous Architectures for Implementing Scalable Cloud Services - Evan Co...
Asynchronous Architectures for Implementing Scalable Cloud Services - Evan Co...Asynchronous Architectures for Implementing Scalable Cloud Services - Evan Co...
Asynchronous Architectures for Implementing Scalable Cloud Services - Evan Co...
 

More from Konrad Malawski

More from Konrad Malawski (10)

Akka Typed (quick talk) - JFokus 2018
Akka Typed (quick talk) - JFokus 2018Akka Typed (quick talk) - JFokus 2018
Akka Typed (quick talk) - JFokus 2018
 
Krakow communities @ 2016
Krakow communities @ 2016Krakow communities @ 2016
Krakow communities @ 2016
 
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK
100th SCKRK Meeting - best software engineering papers of 5 years of SCKRK
 
Zen of Akka
Zen of AkkaZen of Akka
Zen of Akka
 
Distributed Consensus A.K.A. "What do we eat for lunch?"
Distributed Consensus A.K.A. "What do we eat for lunch?"Distributed Consensus A.K.A. "What do we eat for lunch?"
Distributed Consensus A.K.A. "What do we eat for lunch?"
 
Open soucerers - jak zacząć swoją przygodę z open source
Open soucerers - jak zacząć swoją przygodę z open sourceOpen soucerers - jak zacząć swoją przygodę z open source
Open soucerers - jak zacząć swoją przygodę z open source
 
HBase RowKey design for Akka Persistence
HBase RowKey design for Akka PersistenceHBase RowKey design for Akka Persistence
HBase RowKey design for Akka Persistence
 
Scalding - the not-so-basics @ ScalaDays 2014
Scalding - the not-so-basics @ ScalaDays 2014Scalding - the not-so-basics @ ScalaDays 2014
Scalding - the not-so-basics @ ScalaDays 2014
 
DDDing Tools = Akka Persistence
DDDing Tools = Akka PersistenceDDDing Tools = Akka Persistence
DDDing Tools = Akka Persistence
 
Akka persistence == event sourcing in 30 minutes
Akka persistence == event sourcing in 30 minutesAkka persistence == event sourcing in 30 minutes
Akka persistence == event sourcing in 30 minutes
 

Recently uploaded

Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 

Recently uploaded (20)

Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří Karpíšek
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
THE BEST IPTV in GERMANY for 2024: IPTVreel
THE BEST IPTV in  GERMANY for 2024: IPTVreelTHE BEST IPTV in  GERMANY for 2024: IPTVreel
THE BEST IPTV in GERMANY for 2024: IPTVreel
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
 

Reactive integrations with Akka Streams

  • 1. akka streams Reactive Integrations with that just work™ Johan Andrén Konrad Malawski
  • 3. Konrad `ktoso` Malawski Akka Team, Reactive Streams TCK, Persistence, HTTP
  • 4. Make building powerful concurrent & distributed applications simple. Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM
  • 5. Actors – simple & high performance concurrency Cluster / Remoting – location transparency, resilience Cluster tools – and more prepackaged patterns Streams – back-pressured stream processing Persistence – Event Sourcing HTTP – complete, fully async and reactive HTTP Server Official Kafka, Cassandra, DynamoDB integrations, tons more in the community Complete Java & Scala APIs for all features What’s in the toolkit?
  • 6.
  • 8. akka streams Asynchronous back pressured stream processing Source Sink Flow
  • 9. akka streams Asynchronous back pressured stream processing Source Sink (possible) asynchronous boundaries Flow
  • 10. akka streams Asynchronous back pressured stream processing Source Sink 10 msg/s 1 msg/s OutOfMemoryError!! Flow
  • 11. akka streams Asynchronous back pressured stream processing Source Sink 10 msg/s 1 msg/s hand me 3 morehand me 3 more 1 msg/s Flow
  • 12. akka streams Not only linear streams Source SinkFlow Source Sink Flow Flow
  • 13. Reactive Streams Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments as well as network protocols http://www.reactive-streams.org
  • 14. Part of JDK 9 java.util.concurrent.Flow
  • 15. Reactive Streams RS Library A RS library B async boundary
  • 16. Reactive Streams RS Library A RS library B async boundary Make building powerful concurrent & distributed applications simple.
  • 17. The API Akka Streams Complete and awesome Java and Scala APIs (Just like everything in Akka)
  • 18. Akka Streams in 20 seconds: Source<Integer, NotUsed> source = null;
 
 Flow<Integer, String, NotUsed> flow =
 Flow.<Integer>create().map((Integer n) -> n.toString());
 
 Sink<String, CompletionStage<Done>> sink =
 Sink.foreach(str -> System.out.println(str));
 
 RunnableGraph<NotUsed> runnable = source.via(flow).to(sink);
 
 runnable.run(materializer);

  • 19. Akka Streams in 20 seconds: CompletionStage<String> firstString =
 Source.single(1)
 .map(n -> n.toString())
 .runWith(Sink.head(), materializer);

  • 20. Source.single(1).map(i -> i.toString).runWith(Sink.head()) // types: _ Source<Int, NotUsed> Flow<Int, String, NotUsed> Sink<String, CompletionStage<String>> Akka Streams in 20 seconds:
  • 21. Source.single(1).map(i -> i.toString).runWith(Sink.head()) // types: _ Source<Int, NotUsed> Flow<Int, String, NotUsed> Sink<String, CompletionStage<String>> Akka Streams in 20 seconds:
  • 27. What is “materialization” really? Check out the “Implementing an akka-streams materializer for big data” talk later today.
  • 28. AlpakkaA community for Streams connectors http://blog.akka.io/integrations/2016/08/23/intro-alpakka
  • 29. Alpakka – a community for Stream connectors Threading & Concurrency in Akka Streams Explained (part I) Mastering GraphStages (part I, Introduction) Akka Streams Integration, codename Alpakka A gentle introduction to building Sinks and Sources using GraphStage APIs (Mastering GraphStages, Part II) Writing Akka Streams Connectors for existing APIs Flow control at the boundary of Akka Streams and a data provider Akka Streams Kafka 0.11
  • 30. Alpakka – a community for Stream connectors Existing examples: MQTT AMQP Streaming HTTP Streaming TCP Streaming FileIO Cassandra Queries “Reactive Kafka” (akka-stream-kafka) S3, SQS & other Amazon APIs Streaming JSON Streaming XML …
  • 31. Alpakka – a community for Stream connectors Demo
  • 32. Alpakka – a community for Stream connectors Demo
  • 33. Akka Streams & HTTP streams & HTTP
  • 34. A core feature not obvious to the untrained eye…! Akka Streams / HTTP Quiz time! TCP is a ______ protocol?
  • 35. A core feature not obvious to the untrained eye…! Akka Streams / HTTP Quiz time! TCP is a STREAMING protocol!
  • 36. Streaming in Akka HTTP http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html HttpServer as a: Flow[HttpRequest, HttpResponse]
  • 37. Streaming in Akka HTTP HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _] http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
  • 38. Streaming in Akka HTTP HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _] Websocket connection as a: Flow[ws.Message, ws.Message] http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
  • 39. It’s turtles buffers all the way down! xkcd.com
  • 42. Streaming from Akka HTTP No demand from TCP = No demand upstream = Source won’t generate tweets => Bounded memory stream processing! Demo
  • 43. Streaming from Akka HTTP (Java) public static void main(String[] args) { final ActorSystem system = ActorSystem.create(); final Materializer materializer = ActorMaterializer.create(system); final Http http = Http.get(system); final Source<Tweet, NotUsed> tweets = Source.repeat(new Tweet("Hello world")); final Route tweetsRoute = path("tweets", () -> completeWithSource(tweets, Jackson.marshaller(), EntityStreamingSupport.json()) ); final Flow<HttpRequest, HttpResponse, NotUsed> handler = tweetsRoute.flow(system, materializer); http.bindAndHandle(handler, ConnectHttp.toHost("localhost", 8080), materializer ); System.out.println("Running at http://localhost:8080"); }
  • 44. Streaming from Akka HTTP (Java) public static void main(String[] args) { final ActorSystem system = ActorSystem.create(); final Materializer materializer = ActorMaterializer.create(system); final Http http = Http.get(system); final Source<Tweet, NotUsed> tweets = Source.repeat(new Tweet("Hello world")); final Route tweetsRoute = path("tweets", () -> completeWithSource(tweets, Jackson.marshaller(), EntityStreamingSupport.json()) ); final Flow<HttpRequest, HttpResponse, NotUsed> handler = tweetsRoute.flow(system, materializer); http.bindAndHandle(handler, ConnectHttp.toHost("localhost", 8080), materializer ); System.out.println("Running at http://localhost:8080"); }
  • 45. Streaming from Akka HTTP (Scala) object Example extends App with SprayJsonSupport with DefaultJsonProtocol { import akka.http.scaladsl.server.Directives._ implicit val system = ActorSystem() implicit val mat = ActorMaterializer() implicit val jsonRenderingMode = EntityStreamingSupport.json() implicit val TweetFormat = jsonFormat1(Tweet) def tweetsStreamRoutes = path("tweets") { complete { Source.repeat(Tweet("")) } } Http().bindAndHandle(tweetsStreamRoutes, "127.0.0.1", 8080) System.out.println("Running at http://localhost:8080"); }
  • 46. Next steps for Akka Completely new Akka Remoting (goal: 700.000+ msg/s (!)), (it is built using Akka Streams, Aeron). More integrations for Akka Streams stages, project Alpakka. Reactive Kafka polishing with SoftwareMill, Krzysiek Ciesielski Akka Typed progressing again, likely towards 3.0. Akka HTTP 2.0 Proof of Concept in progress. Collaboration with Reactive Sockets
  • 47. Ready to adopt on prod?
  • 49. Akka <3 contributions Easy to contribute tickets: https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22 Akka Stream Contrib https://github.com/akka/akka-stream-contrib Mailing list: https://groups.google.com/group/akka-user Public chat rooms: http://gitter.im/akka/dev developing Akka http://gitter.im/akka/akka using Akka
  • 51. Further reading: Reactive Streams: reactive-streams.org Akka documentation: akka.io/docs Free O’Reilly report – very out soon. Example Sources: ktoso/akka-streams-alpakka-talk-demos-2016 Get involved: sources: github.com/akka/akka mailing list: akka-user @ google groups gitter channel: https://gitter.im/akka/akka Contact: Konrad ktoso@lightbend.com Malawski http://kto.so / @ktosopl