As the number of systems within an IT infrastructure increases, the number of integrations needed by enterprises also multiplies. Recognizing that the old times of overnight file exchanges are no longer meeting real-time demands, a well-organized enterprise integration strategy is a critical success factor when your systems need to be connected all day. In this webinar with Enno Runne, Tech Lead for Alpakka at Lightbend, Inc., we’ll look at why integrations should be viewed as streams of data, and how Alpakka—a Reactive Enterprise Integration library for Java and Scala based on Reactive Streams and Akka—fits perfectly for today’s demands on system integrations. Specifically, we will review: * How Alpakka brings streaming data flows directly to the surface, utilizing the features of Akka to tame the complexity of streams. * Supported connectors for Amazon Web Services, Microsoft Azure, and Google Cloud, as well as others for event sourcing/persistence/DB technologies and traditional interfaces like FTP, HTTP, etc. * A deeper look into the use cases for Alpakka’s most utilized interfaces to popular technologies like Apache Kafka, MQTT, and MongoDB.

2. • Alpakka Tech Lead @ Lightbend
• Stockholm, Sweden
Enno Runne

3. Quick Intro to Lightbend
2017
Acquired OpsClarity
6m+ OSS Downloads/month
Launched Fast Data Platform
Investment from IBM
Launched
Lagom Framework
2016
2015
Spark Support
500k+ OSS Downloads/
month
Acquired
Play Framework
Founded by
Martin Odersky, Scala
creator & Jonas Bonér,
Akka creator
2012
20132011
2014
Launched
Reactive Platform
Published the
Reactive Manifesto

4. Enterprise
Integrations

5. Enterprise Integration Patterns
• The classic book on integration
• Gregor Hohpe, Bobby Woolf
• Published 2003
http://www.enterpriseintegrationpatterns.com

7. Reactive
Integrations

8. Listen to
queue
Download
file
Extract data
Store
file

9. Queue
File
source
File-
store
Listen to
queue
Download
file
Extract data
Store
file

10. Listen to
MQTT
topic
Download
file via
HTTP
Parse as
JSON
Store page
content on
AWS S3
MQTT
Broker
Web
server
AWS
S3
1. Queue message
2. Parse JSON
3. Request file
4. File content entity data
5. Store file on S3

11. Data flow
Step 1 Step 4Step 2 Step 3
Buffering data
• May cure the immediate issue
• All buffering is limited

12. Downstream
Source SinkFlow Flow
Streaming with back-pressure
Dynamic push/pull
• Push when downstream is faster
• Pull when upstream is faster

13. Reactive Integrations
Cross-system back-pressure support
is the key thing Reactive Integrations
bring to the table.

14. Reactive Streams
reactive-streams.org

15. Part of JDK 9 (java.util.concurrent.Flow)
http://www.reactive-streams.org/
Compliant libraries allow full interoperability
Reactive Streams
A standard for asynchronous
stream processing with non-
blocking back-pressure.

16. Listen to
MQTT
topic
Download
file via
HTTP
Parse as
JSON
Store page
content on
AWS S3
MQTT
Broker
Web
server
AWS
S3
Streams on multiple levels

17. Akka?
How does streaming relate to
Akka Streams run on top of Akka Actors
Akka takes care of the intricacies of signalling demand

18. • Domain Specific Language to describe streams
• Mechanics to map these stream to be executed on top of actors
• Full support to connect to any Reactive Streams compliant library
Akka Streams

19. Extends Reactive Streams to be used
• for local file access
• over TCP as TCP supports flow control
• for HTTP with Akka HTTP
Akka Streams

20. When to use what abstraction
modeling power
complexity
actors
streams
(completable)
futures
java.concurrency

21. Live
Coding

22. Listen to
MQTT
topic
Download
file via
HTTP
Parse as
JSON
Store page
content on
AWS S3
MQTT
Broker
Web
server
AWS
S3
Example scenario

23. Listen to
MQTT
topic
Download
file via
HTTP
Parse as
JSON
Store page
content on
AWS S3
MQTT
Broker
Web
server
AWS
S3
Example scenario

25. Alpakka is a Reactive Enterprise Integration
library for Java and Scala, based on Reactive
Streams and Akka.
Alpakka
The short version: “Endpoints for Akka Streams”

26. Alpakka
• Typed message interchange,
compiler tracks compatibility
• Back-pressure as specified by
Reactive Streams
• No OSGi support
• Relies on Akka Streams; evolving
rapidly, but many integration
requirements are not covered, yet
• Moving fast, docs split between
Akka and Alpakka sites
My view on Apache Camel vs Alpakka
Apache Camel
• Data is wrapped in Exchange instance,
type can only be inspected
• No back-pressure awareness, can
connect to Reactive Streams compliant
systems
• OSGi support
• Full-featured framework to express
integrations
• Comprehensive docs and books

27. Alpakka connectors for cloud services
Amazon DynamoDB
Amazon Kinesis data
streams & firehose
AWS Lambda
Amazon S3
Amazon SNS
Amazon SQS
Google Cloud Pub/
Sub
Google Firebase
Cloud Messaging
Azure Storage Queue

28. Alpakka connectors for data stores
Elasticsearch

29. Alpakka connectors for messaging
MQTT
(Eclipse Paho)
AMQP
(RabbitMQ)
IronMQ
JMS
Java Messaging Service
Apache Kafka
… not as fancy logos, but very well suited for the streaming approach.

30. Preparations for Alpakka 1.0 include
• structure all modules the same
• improve chances to stay binary compatible
• ensure good test coverage
Help with this is highly appreciated…
The road to Alpakka 1.0

31. Code, Issues, Pull Requests @ Github
• https://github.com/akka/alpakka
• https://github.com/akka/alpakka-kafka
Questions, Discussions
• https://discuss.lightbend.com/c/akka
Alpakka community

32. Thank you!
@ennru
enno.runne@lightbend.com
https://www.lightbend.com/alpakka
@akkateam
Join the Alpakka community
at github.com/akka/alpakka
https://github.com/akka/alpakka-samples