The document discusses non-blocking asynchronous programming with futures and promises in Scala. It explains why non-blocking is important to avoid blocking threads, describes how to compose futures to build asynchronous workflows, and presents a model of building servers as asynchronous functions that return futures. It advocates for building reactive systems with non-blocking services, filters, and asynchronous composition.

1. DON’T BLOCK YOURSELF
Flavio W. Brasil - SoundCloud

2. @flaviowbrasil
@fwbrasil
backend engineer

4. Service compositiongetclump/clump

5. Service composition
Distributed Durable STM
getclump/clump
fwbrasil/activate

6. Service composition
Distributed Durable STM
Sane scala reflection
getclump/clump
fwbrasil/activate
fwbrasil/smirror

7. Service composition
Distributed Durable STM
Sane scala reflection
Type-level validation
getclump/clump
fwbrasil/activate
fwbrasil/smirror
fwbrasil/bond

8. Service composition
Distributed Durable STM
Sane scala reflection
Type-level validation
Type-safe REST
getclump/clump
fwbrasil/activate
fwbrasil/smirror
fwbrasil/bond
fwbrasil/zoot

10. 12 hours of new content/minute

11. 12 hours of new content/minute
350 million users/month

12. 12 hours of new content/minute
350 million users/month
1 outer space user

15. 1) WHY NON-
BLOCKING?
!
2) FUTURES
AND PROMISES
!
3) YOUR
SERVER AS A
FUNCTION

16. PART ONE
!
WHY NON-
BLOCKING?

18. How many requests?

21. TimelineAPI

22. TimelineAPI

23. TimelineAPI

24. TimelineAPI

25. TimelineAPI

26. TimelineAPI

27. TimelineAPI

28. TimelineAPI
Blocking IO

29. TimelineAPI
Blocking IO

30. TimelineAPI
Blocking IO

31. Threads aren't free!

32. Threads aren't free!
3 facts

33. Java uses native threads
1

34. The stack uses memory
2

35. Garbage collection
pressure
3

36. GC pressure
Java uses native threads
The stack uses memory

38. Multiplexing!

44. Waiters aren't free!

50. Thread

51. Thread
EventLoop

52. TimelineAPI

53. TimelineAPI
timeline.stream(u):
List[Event]

54. TimelineAPI

55. TimelineAPI
timeline.stream(u, callback)

56. TimelineAPI
timeline.stream(u):
Future[List[Event]]

57. PART TWO
!
FUTURES AND
PROMISES

58. Future[List[Event]]

59. Reference for an
asynchronous operation
completion
Future[List[Event]]

60. Returns immediately
timeline
.stream(u): Future[List[Event]]

61. future
.poll: Option[Try[List[Event]]]

62. !
val filtered: Future[List[Event]] =
timeline.stream(u).map {
_.filter(_.isTrackPost)
}
Compose a new future

63. !
val filtered: Future[List[Event]] =
timeline.stream(u).map {
_.filter(_.isTrackPost)
}
Compose a new future

64. !
val filtered: Future[List[Event]] =
timeline.stream(u).map {
_.filter(_.isTrackPost)
}
Compose a new future

65. Compose
a new future using
another future
timeline.stream(u).flatMap {
fetchTrackMetadata(_)
}

66. val session =
authenticate(request)
!
val geolocation =
geolocate(request)

67. Parallel
val session =
authenticate(request)
!
val geolocation =
geolocate(request)

68. !
session.join(geolocation).map {
case (session, geo) => doSomething
}
Compose parallel futures

69. !
session.join(geolocation).map {
case (session, geo) => doSomething
}
Compose parallel futures

70. !
val tracks: Seq[Future[Track]] =
ids.map(tracksService.fetch)
!
val collect: Future[Seq[Track]] =
Future.collect(tracks)
Compose parallel futures

71. !
val tracks: Seq[Future[Track]] =
ids.map(tracksService.fetch)
!
val collect: Future[Seq[Track]] =
Future.collect(tracks)
Compose parallel futures

72. Recursive composition

73. Recursive composition
def likes(user: Urn, offset: Int = 0) =
likesService.fetch(user, offset).flatMap {
case page if(page.hasNext) =>
likes(user, page.nextOffset).map {
page.events ++ _
}
case page =>
Future.value(page.events)
}

74. Recursive composition
def likes(user: Urn, offset: Int = 0) =
likesService.fetch(user, offset).flatMap {
case page if(page.hasNext) =>
likes(user, page.nextOffset).map {
page.events ++ _
}
case page =>
Future.value(page.events)
}

75. Recursive composition
def likes(user: Urn, offset: Int = 0) =
likesService.fetch(user, offset).flatMap {
case page if(page.hasNext) =>
likes(user, page.nextOffset).map {
page.events ++ _
}
case page =>
Future.value(page.events)
}

76. Recursive composition
def likes(user: Urn, offset: Int = 0) =
likesService.fetch(user, offset).flatMap {
case page if(page.hasNext) =>
likes(user, page.nextOffset).map {
page.events ++ _
}
case page =>
Future.value(page.events)
}

77. Recursive composition
def likes(user: Urn, offset: Int = 0) =
likesService.fetch(user, offset).flatMap {
case page if(page.hasNext) =>
likes(user, page.nextOffset).map {
page.events ++ _
}
case page =>
Future.value(page.events)
}

78. Recursive composition
def likes(user: Urn, offset: Int = 0) =
likesService.fetch(user, offset).flatMap {
case page if(page.hasNext) =>
likes(user, page.nextOffset).map {
page.events ++ _
}
case page =>
Future.value(page.events)
}

79. !
timeline.stream(u).flatMap { events =>
Future.collect(Seq(
fetchTracks(events),
fetchPlaylists(events),
fetchComments(events),
fetchUsers(events)))
}

80. !
timeline.stream(u).flatMap { events =>
Future.collect(Seq(
fetchTracks(events),
fetchPlaylists(events),
fetchComments(events),
fetchUsers(events)))
}

81. Timeout
!
timeline.stream(u).flatMap { events =>
Future.collect(Seq(
fetchTracks(events),
fetchPlaylists(events),
fetchComments(events),
fetchUsers(events)))
}.within(2.seconds)

82. Cancellation
!
timeline.stream(u).flatMap { events =>
Future.collect(Seq(
fetchTracks(events),
fetchPlaylists(events),
fetchComments(events),
fetchUsers(events)))
}.within(2.seconds)

83. Local values

84. !
object Context {
val requestId = new Local[Int]
}
!
def endpoint(request: Request) =
callService1
.map(applyATransformation(_))
.flatMap(callService2)
timeline.stream(u, Context.requestId())
Local values

85. !
object Context {
val requestId = new Local[Int]
}
!
def endpoint(request: Request) =
callService1
.map(applyATransformation(_))
.flatMap(callService2)
timeline.stream(u, Context.requestId())
Local values

86. !
object Context {
val requestId = new Local[Int]
}
!
def endpoint(request: Request) =
callService1
.map(applyATransformation(_))
.flatMap(callService2)
timeline.stream(u, Context.requestId())
Local values

87. Promise[Stream]

88. A promise is a
writable future
Promise[Stream]

89. val promise = Promise[List[Event]]()
promise.setValue(List())

90. Not common to use
val promise = Promise[List[Event]]()
promise.setValue(List())

91. PART THREE
!
YOUR SERVER
AS A FUNCTION

93. The tripod

94. Futures
Services
Filters

95. Futures

96. Services

97. Services
type Service[Req, Rep]
= Req => Future[Rep]

98. Services
Asynchronous functions
that represent systems
boundaries
type Service[Req, Rep]
= Req => Future[Rep]

99. Services
Symmetric and uniform
API for clients and servers
type Service[Req, Rep]
= Req => Future[Rep]

100. Services
val client: Service[HttpReq, HttpRep] = …
!
val f: Future[HttpRep] = client(HttpReq("/"))

101. Services
val client: Service[HttpReq, HttpRep] = …
!
val f: Future[HttpRep] = client(HttpReq("/"))

102. Services
val client: Service[HttpReq, HttpRep] = …
!
val f: Future[HttpRep] = client(HttpReq("/"))
val server: Service[HttpReq, HttpRep] = …
!
Http.serve(":80", service)

103. Services
val client: Service[HttpReq, HttpRep] = …
!
val f: Future[HttpRep] = client(HttpReq("/"))
val server: Service[HttpReq, HttpRep] = …
!
Http.serve(":80", service)

104. one-liner proxy

105. Http.serve(“:81", Http.newClient(“127.0.0.1:80”))
one-liner proxy

106. Filters
type Filter[Req, Rep] =
(Req, Service[Req, Rep]) => Future[Rep]

107. Filters
type Filter[Req, Rep] =
(Req, Service[Req, Rep]) => Future[Rep]
Application agnostic concerns
(timeout, retry, etc)

108. Filters
class TimeoutFilter[Req, Rep]
extends SimpleFilter[Req, Rep] {
def apply(req: Req, service: Service[Req, Rep]) =
service(req).within(2.seconds)
}

109. Filters
val serviceWithTimeout =
new TimeoutFilter andThen service
class TimeoutFilter[Req, Rep]
extends SimpleFilter[Req, Rep] {
def apply(req: Req, service: Service[Req, Rep]) =
service(req).within(2.seconds)
}

111. Not only
a tripod

112. Not only
NIO

113. - Connection pooling

114. - Connection pooling
- Load-balancers

115. - Connection pooling
- Load-balancers
- Distributed tracing

116. - Connection pooling
- Load-balancers
- Distributed tracing
- Service discovery

117. - Connection pooling
- Load-balancers
- Distributed tracing
- Service discovery
- Metrics

119. REACTIVE!!!
!!!!1111

120. - Failover strategies

121. - Failover strategies
- Failure detectors

122. - Failover strategies
- Failure detectors
- Dynamic cluster

123. - Failover strategies
- Failure detectors
- Dynamic cluster
- Back-pressure

124. MUX

125. MUX
GC
avoidance

126. Failure detector

127. kraken example
Errors / second
Failure detector

128. kraken example
Errors / second
Failure detector

129. CONCLUSIONS

131. - Can't use scala futures
!
!

132. - Can't use scala futures
- slow adoption of new
scala versions

133. - Can't use scala futures
- slow adoption of new
scala versions
- netty 3

134. - Can't use scala futures
- slow adoption of new
scala versions
- netty 3
- Internals are complex

135. - Can't use scala futures
- slow adoption of new
scala versions
- netty 3
- Internals are complex
- Mysql driver is limited

136. Happy user

137. - Really good toolchain
-
-
-

138. - Really good toolchain
- Many protocols
-
-

139. - Really good toolchain
- Many protocols
- Active community
-

140. - Really good toolchain
- Many protocols
- Active community
- Battle tested

141. DON’T BLOCK YOURSELF
Flavio W. Brasil - SoundCloud