This document provides an introduction to Akka.NET Streams and Reactive Streams. It discusses key concepts like observables, async enumerables, and reactive streams. It also demonstrates how to build workflows with Akka.NET streams, including examples of building a TCP server. The document introduces core Akka.NET streams concepts like sources, flows, and sinks, and how they compose together in a runnable graph. It also covers testing streams with probes and materialization.

1. AKKA.NET STREAMS
AND REACTIVE STREAMS

2. INTRODUCTION
Bartosz Sypytkowski
▪ @Horusiath
▪ b.sypytkowski@gmail.com
▪ bartoszsypytkowski.com

3.  Observables, AsyncEnumerable
 Reactive Streams
 Akka.Streams for building workflows
 Doing I/O
 Evensourcing with Akka.Streams
 Future?
AGENDA

4. OBSERVABLES
public interface IObserver<in T>
{
void OnNext(T item);
void OnError(Exception e);
void OnCompleted();
}
public interface IObservable<out T>
{
IDisposable Subscribe(
IObserver<T> observer);
}

5. OBSERVABLES STAGESTAGE
OnNext(T)

6. PROBLEM
NO BACKPRESSURE

7. OBSERVABLES STAGESTAGE
OnNext(T)
100 RPS 10 RPS

8. ASYNC
ENUMERABLES
public interface IAsyncEnumerable<out T>
{
IAsyncEnumerator<T> GetAsyncEnumerator();
}
public interface IAsyncEnumerator<out T> :
IAsyncDisposable
{
T Current { get; }
ValueTask<bool> MoveNextAsync();
}
public interface IAsyncDisposable
{
ValueTask DisposeAsync();
}

9. ASYNC
ENUMERABLE
PULLING
STAGESTAGE
MoveNext
(Value)Task<T>

10. WHAT IF REQUESTS ARE
EXPENSIVE?

11. ASYNC
ENUMERABLE
OVER I/O
STAGESTAGE
I/O

12. JUST USE BUFFERING

13. ASYNC
ENUMERABLE
OVER I/O
STAGESTAGE
I/O
BUFFER
BUFFER

14. PROBLEM
THIS SH**T DOES NOT COMPOSE

15. ASYNC
ENUMERABLE
PROBLEM WITH CAPACITY
PROPAGATION
STAGE
Buffer[M]
STAGE
Buffer[N]
STAGE
Buffer[K]
What is the M?

16. REACTIVE
STREAMS
public interface ISubscription
{
void Request(long demand);
void Cancel();
}
public interface ISubscriber<in T>
{
void OnSubscribe(ISubscription s);
void OnNext(T item);
void OnError(Exception e);
void OnComplete();
}
public interface IPublisher<out T>
{
void Subscribe(ISubscriber<T> s);
}
public interface IProcessor<in TIn, out TOut> :
ISubscriber<TIn>, IPublisher<TOut>
{ }

17. REACTIVE
STREAMS
DEMAND AS PART OF
PROTOCOL
STAGE
Buffer[M]
STAGE
Buffer[N]
Demand(M)
Send(X)

18. REACTIVE
STREAMS
DEMAND PROPAGATION
STAGE
50 RPS
Req(30)
STAGE
20 RPS
STAGE
30 RPS
Req(20)

19. REACTIVE
STREAMS
DEMAND COMPOSITION
SCALLING
30 RPS
30 RPS
50 RPS
50 RPS
50 RPS
Req(30)
Req(30)
Req(20)
Req(20)
Req(20)

20. REACTIVE STREAMS ARE
UNIVERSALLY COMPOSABLE

21. REACTIVE
STREAMS
Publisher Subscriber
Subscribe

22. REACTIVE
STREAMS
Publisher Subscriber
Subscribe
OnSubscribe

23. REACTIVE
STREAMS
Publisher Subscriber
Subscribe
OnSubscribe
Request(3)

24. REACTIVE
STREAMS
Publisher Subscriber
Subscribe
OnSubscribe
Request(3)
OnNext

25. REACTIVE
STREAMS
Publisher Subscriber
Subscribe
OnSubscribe
Request(3)
OnNext
OnNext

26. REACTIVE
STREAMS
Publisher Subscriber
Subscribe
OnSubscribe
Request(3)
OnNext
OnNext
Request(2)

27. REACTIVE
STREAMS
Publisher Subscriber
Subscribe
OnSubscribe
Request(3)
OnNext
OnNext
Request(2)
OnNext

28. REACTIVE
STREAMS
Publisher Subscriber
Subscribe
OnSubscribe
Request(3)
OnNext
OnNext
Request(2)
OnNext
OnNext

29. REACTIVE
STREAMS
Publisher Subscriber
Subscribe
OnSubscribe
Request(3)
OnNext
OnNext
Request(2)
OnNext
OnNext
OnComplete

30. OBSERVABLES
AS REACTIVE STREAMS

31. AS REACTIVE STREAMS
OBSERVABLES
Publisher Subscriber
Subscribe
OnSubscribe
Request(long.MaxValue)
OnNext
OnNext
OnNext
OnNext
OnComplete

32. ASYNC ENUMERABLES
AS REACTIVE STREAMS

33. AS REACTIVE STREAMS
ASYNC
ENUMERABLES
Publisher Subscriber
Subscribe
OnSubscribe
Request(1)
OnNext
OnNext
OnComplete
Request(1)
Request(1)

34. BUT… WE COULD IMPLEMENT
THIS SOLUTION USING TASKS

35. WE COULD IMPLEMENT
TASKS USING THIS SOLUTION

36. AS REACTIVE STREAMS
TASKS
Publisher Subscriber
Subscribe
OnSubscribe
Request(1)
OnNext
OnComplete

37. RSOCKET
REACTIVE STREAMS OVER THE NETWORK

38. REACTIVE.STREAMS.TCK
TEST COMPATIBILITY KIT

39. AKKA.NET STREAMS
WHERE ACTORS AND STREAMS COLLIDE

40. AKKA.NET
STREAMS
101 Source<Seller, NotUsed> source = GetSource(ConnectionString);
Sink<Tuple<string, long>, Task> sink =
Sink.ForEach<Tuple<string, long>>(x =>
Console.WriteLine($"{x.Item2}t{x.Item1}"));
await source
.Collect(seller => seller.name)
.ZipWithIndex()
.ToMaterialized(sink, Keep.Right)
.Run(materializer);

41. AKKA.NET
STREAMS
101 Source<Seller, NotUsed> source = GetSource(ConnectionString);
Sink<Tuple<string, long>, Task> sink =
Sink.ForEach<Tuple<string, long>>(x =>
Console.WriteLine($"{x.Item2}t{x.Item1}"));
await source
.Collect(seller => seller.name)
.ZipWithIndex()
.ToMaterialized(sink, Keep.Right)
.Run(materializer);
Producer of events

42. AKKA.NET
STREAMS
101 Source<Seller, NotUsed> source = GetSource(ConnectionString);
Sink<Tuple<string, long>, Task> sink =
Sink.ForEach<Tuple<string, long>>(x =>
Console.WriteLine($"{x.Item2}t{x.Item1}"));
await source
.Collect(seller => seller.name)
.ZipWithIndex()
.ToMaterialized(sink, Keep.Right)
.Run(materializer);
Consumer of events

43. AKKA.NET
STREAMS
101 Source<Seller, NotUsed> source = GetSource(ConnectionString);
Sink<Tuple<string, long>, Task> sink =
Sink.ForEach<Tuple<string, long>>(x =>
Console.WriteLine($"{x.Item2}t{x.Item1}"));
await source
.Collect(seller => seller.name)
.ZipWithIndex()
.ToMaterialized(sink, Keep.Right)
.Run(materializer);
Materialized values aka. results

44. AKKA.NET
STREAMS
101 Source<Seller, NotUsed> source = GetSource(ConnectionString);
Sink<Tuple<string, long>, Task> sink =
Sink.ForEach<Tuple<string, long>>(x =>
Console.WriteLine($"{x.Item2}t{x.Item1}"));
await source
.Collect(seller => seller.name)
.ZipWithIndex()
.ToMaterialized(sink, Keep.Right)
.Run(materializer);
Graph evaluation

45. AKKA.NET STREAMS
CORE CONCEPTS
Source<TIn, M> Flow<TIn, TOut, M> Sink<TOut, M>

46. AKKA.NET STREAMS
CORE CONCEPTS
Source<TIn, M> Flow<TIn, TOut, M> Sink<TOut, M>
Only produces data

47. AKKA.NET STREAMS
CORE CONCEPTS
Source<TIn, M> Flow<TIn, TOut, M> Sink<TOut, M>
Transforms data

48. AKKA.NET STREAMS
CORE CONCEPTS
Source<TIn, M> Flow<TIn, TOut, M> Sink<TOut, M>
Only consumes data

49. AKKA.NET STREAMS
CORE CONCEPTS
Source<TIn, M> Flow<TIn, TOut, M> Sink<TOut, M>
RunnableGraph<M>

50. AKKA.NET
STREAMS
101 Source<Seller, NotUsed> source = GetSource(ConnectionString);
Sink<Tuple<string, long>, Task> sink =
Sink.ForEach<Tuple<string, long>>(x =>
Console.WriteLine($"{x.Item2}t{x.Item1}"));
await source
.Collect(seller => seller.name)
.ZipWithIndex()
.ToMaterialized(sink, Keep.Right)
.Run(materializer);
RunnableGraph<Task>

51. MATERIALIZATION
USING AKKA.NET INTERFACES
RunnableGraph<M>
+
IMaterializer
=>
Materialized Value

52. MATERIALIZATION
USING .NET INTERFACES
IQueryable<T>
+
QueryProvider
=>
Materialized Value

53. PROBLEMS WITH
QUERYABLE
MATERIALIZATION
1. Implicit query provider =
accidental materialization
var sellerPage = (
from seller in dbContext.Sellers
orderby seller.SellerId
select seller)
.ToList()
.Skip(page * PageSize).Take(PageSize);

54. PROBLEMS WITH
QUERYABLE
MATERIALIZATION
1. Implicit query provider =
accidental materialization
2. No materialized value = hard
to get multiple results
var sellerPage = await (
from seller in dbContext.Sellers
orderby seller.SellerId
select seller)
.Skip(page * PageSize).Take(PageSize)
.ToListAsync();
var total = await
dbContext.Sellers.CountAsync();

55. WHY USE THAT INSTEAD OF
TPL DATAFLOW?

56. 1. Imperative
2. Most common composition
primitives.
AKKA.NET
STREAMS
TPL DATAFLOW
1. Declarative (like LINQ)
2. Ton of utility functions
(batching, rate limiting etc.)
3. Universal abstraction

57. BUT IS IT FAST?

58. ALSO REMEMBER
THERE ARE LIES, BIG LIES AND BENCHMARKS
WANT TO BE SURE? TRY IT FOR YOURSELF

59. AKKA.NET
STREAMS
TESTING
public class PulseSpec : Akka.TestKit.Xunit2.TestKit
{
private readonly TimeSpan _pulseInterval = TimeSpan.FromMilliseconds(20);
[Fact]
public async Task Pulse_should_signal_demand_once_every_interval()
{
var (probe, task) = this.SourceProbe<int>()
.Via(new Pulse<int>(_pulseInterval))
.ToMaterialized(Sink.Seq<int>(), Keep.Both)
.Run(Sys.Materializer());
probe.SendNext(1);
probe.ExpectNoMsg(_pulseInterval);
probe.SendNext(2);
probe.ExpectNoMsg(_pulseInterval);
probe.SendComplete();
var result = await task;
result.ShouldBeEquivalentTo(new[] {1, 2}, o => o.WithStrictOrdering());
}
}

60. AKKA.NET
STREAMS
TESTING
public class PulseSpec : Akka.TestKit.Xunit2.TestKit
{
private readonly TimeSpan _pulseInterval = TimeSpan.FromMilliseconds(20);
[Fact]
public async Task Pulse_should_signal_demand_once_every_interval()
{
var (probe, task) = this.SourceProbe<int>()
.Via(new Pulse<int>(_pulseInterval))
.ToMaterialized(Sink.Seq<int>(), Keep.Both)
.Run(Sys.Materializer());
probe.SendNext(1);
probe.ExpectNoMsg(_pulseInterval);
probe.SendNext(2);
probe.ExpectNoMsg(_pulseInterval);
probe.SendComplete();
var result = await task;
result.ShouldBeEquivalentTo(new[] {1, 2}, o => o.WithStrictOrdering());
}
}
Manages Akka.NET runtime for testing purposes

61. AKKA.NET
STREAMS
TESTING
public class PulseSpec : Akka.TestKit.Xunit2.TestKit
{
private readonly TimeSpan _pulseInterval = TimeSpan.FromMilliseconds(20);
[Fact]
public async Task Pulse_should_signal_demand_once_every_interval()
{
var (probe, task) = this.SourceProbe<int>()
.Via(new Pulse<int>(_pulseInterval))
.ToMaterialized(Sink.Seq<int>(), Keep.Both)
.Run(Sys.Materializer());
probe.SendNext(1);
probe.ExpectNoMsg(_pulseInterval);
probe.SendNext(2);
probe.ExpectNoMsg(_pulseInterval);
probe.SendComplete();
var result = await task;
result.ShouldBeEquivalentTo(new[] {1, 2}, o => o.WithStrictOrdering());
}
}
Feed stream in controlled fashion

62. AKKA.NET
STREAMS
TESTING
public class PulseSpec : Akka.TestKit.Xunit2.TestKit
{
private readonly TimeSpan _pulseInterval = TimeSpan.FromMilliseconds(20);
[Fact]
public async Task Pulse_should_signal_demand_once_every_interval()
{
var (probe, task) = this.SourceProbe<int>()
.Via(new Pulse<int>(_pulseInterval))
.ToMaterialized(Sink.Seq<int>(), Keep.Both)
.Run(Sys.Materializer());
probe.SendNext(1);
probe.ExpectNoMsg(_pulseInterval);
probe.SendNext(2);
probe.ExpectNoMsg(_pulseInterval);
probe.SendComplete();
var result = await task;
result.ShouldBeEquivalentTo(new[] {1, 2}, o => o.WithStrictOrdering());
}
}
Close the stream

63. LET’S BUILD SOMETHING
MAYBE A TCP SERVER?

64. AKKA.NET
STREAMS
TCP
SERVER
var prefix = ByteString.FromString("Hello, ");
Flow<ByteString, ByteString, NotUsed> handler =
Framing.Delimiter(ByteString.FromString("n"), 4000, true)
.Select(message => prefix + message);
var binding = await
system.TcpStream()
.Bind(“127.0.0.1", 5000)
.To(Sink.ForEach<Tcp.IncomingConnection>(incoming =>
{
Console.WriteLine($"Client connected: {incoming.RemoteAddress}");
incoming.Flow.Join(handler).Run(materializer);
}))
.Run(materializer);
// Shutdown server
await binding.Unbind();

65. var prefix = ByteString.FromString("Hello, ");
Flow<ByteString, ByteString, NotUsed> handler =
Framing.Delimiter(ByteString.FromString("n"), 4000, true)
.Select(message => prefix + message);
var binding = await
system.TcpStream()
.Bind(“127.0.0.1", 5000)
.To(Sink.ForEach<Tcp.IncomingConnection>(incoming =>
{
Console.WriteLine($"Client connected: {incoming.RemoteAddress}");
incoming.Flow.Join(handler).Run(materializer);
}))
.Run(materializer);
// Shutdown server
await binding.Unbind();
AKKA.NET
STREAMS
TCP
SERVER
Bind server to IP endpoint

66. var prefix = ByteString.FromString("Hello, ");
Flow<ByteString, ByteString, NotUsed> handler =
Framing.Delimiter(ByteString.FromString("n"), 4000, true)
.Select(message => prefix + message);
var binding = await
system.TcpStream()
.Bind(“127.0.0.1", 5000)
.To(Sink.ForEach<Tcp.IncomingConnection>(incoming =>
{
Console.WriteLine($"Client connected: {incoming.RemoteAddress}");
incoming.Flow.Join(handler).Run(materializer);
}))
.Run(materializer);
// Shutdown server
await binding.Unbind();
AKKA.NET
STREAMS
TCP
SERVER
Source<Tcp.IncomingConnection, Task<Tcp.ServerBinding>>

67. var prefix = ByteString.FromString("Hello, ");
Flow<ByteString, ByteString, NotUsed> handler =
Framing.Delimiter(ByteString.FromString("n"), 4000, true)
.Select(message => prefix + message);
var binding = await
system.TcpStream()
.Bind(“127.0.0.1", 5000)
.To(Sink.ForEach<Tcp.IncomingConnection>(incoming =>
{
Console.WriteLine($"Client connected: {incoming.RemoteAddress}");
incoming.Flow.Join(handler).Run(materializer);
}))
.Run(materializer);
// Shutdown server
await binding.Unbind();
AKKA.NET
STREAMS
TCP
SERVER
Connection handler logic

68. var prefix = ByteString.FromString("Hello, ");
Flow<ByteString, ByteString, NotUsed> handler =
Framing.Delimiter(ByteString.FromString("n"), 4000, true)
.Select(message => prefix + message);
var binding = await
system.TcpStream()
.Bind(“127.0.0.1", 5000)
.To(Sink.ForEach<Tcp.IncomingConnection>(incoming =>
{
Console.WriteLine($"Client connected: {incoming.RemoteAddress}");
incoming.Flow.Join(handler).Run(materializer);
}))
.Run(materializer);
// Shutdown server
await binding.Unbind();
AKKA.NET
STREAMS
TCP
SERVER
Efficient byte management

69. BYTESTRING
1. Low-alloc byte manipulation
API.
2. Immutable, indexable and
composable sequence of bytes.
3. Internally a collection of
chunks of bytes.
ByteString
0110010 0111110 010111011
0100110010000001 010

70. AKKA.NET
STREAMS
EVENT
SOURCING
Actor
A B C D E F G H I
Event log
Source SourceSource
J

71. STREAM REFS
MAKE STREAM CONSTRUCTION DISTRIBUTED
AND TRANSPARENT

72. AKKA.NET
STREAM
REFS
NODE 1
CREATE SOURCE
REF ON ONE SIDE
RunnableGraph<SourceRef<T>>
NODE 2

73. AKKA.NET
STREAM
REFS
NODE 1
MATERIALIZE
STREAM
NODE 2
SourceRef<T>

74. AKKA.NET
STREAM
REFS
NODE 1
SEND SOURCE REF
OVER THE NETWORK
NODE 2

75. AKKA.NET
STREAM
REFS
NODE 1
MATERIALIZE AS
PART OF GRAPH ON
ANOTHER NODE
NODE 2

76. WHAT WE
HAVEN’T
TALKED
ABOUT
1. Building graphs with GraphDSL
2. Creating custom stages
3. Bidirectional flows

77.  RSocket site: http://rsocket.io/
 Akka.NET Streams documentation: https://getakka.net/articles/streams/introduction.html
 Alpakka – Akka.NET connectors: https://github.com/AkkaNetContrib/Alpakka
REFERENCES

78. THANK YOU