The document discusses advancements in fast data architecture and microservices, highlighting various tools and methodologies, such as Akka and streaming engines, to improve performance in real-time applications like financial services. It details specific case studies, including Capital One and Verizon, illustrating successful transitions from monolithic to microservices architectures. Additionally, the document outlines common anti-patterns in using Akka, focusing on best practices for effective implementation and system performance.

1. at

2. “Top microservices and
fast data experts in the world.”

3. How Events Are Reshaping Modern Systems
Jonas Bonér
Lightbend CTO

14. The future is a function of the past - A. P. Robertson
The (local) present is a merge function of
multiple concurrent pasts - Jonas Bonér
val newLocalPresent = observedPasts
foldLeft(oldLocalPresent) { _ merge _ }

21. Stream All the Things!!
Dean Wampler
VP Fast Data Engineering
Lightbend

22. Hadoop classic batch architecture

23. Streaming, Fast Data Architecture

25. Latency Use case Suggested engine
Picoseconds to a few microseconds
(real-time)
Space X rocket
Custom hardware (FPGA), no JVM or
anything on top
Less than 100 microseconds Financial services Actors on the JVM
Less than 10 milliseconds Authorising credit cards
Fast data streaming like Flink, Akka
Streams, Kafka Streams
Less than 100 milliseconds Dashboards, Machine learning
We can think of micro-batches, so to
process records in bulk. Spark /
Streaming SQL
Less than 1 second to a minute ETL (Extract, Transform, Load) Mini-batches (spark streaming)
More than a few minutes
Check for fake reviews on a travel
website
Batch jobs
Latency cases to choose a streaming engine

26. Other choosing criteria
• Volume of data?
• Process records individually or in bulk?
• How do I want to deploy my app?
Overview of streaming engines and relation
with micro services

27. “Dean Wampler
StreamAllTheThings.pdf”
http://bit.ly/lightbend-fast-data
Search for:

29. Using Akka, Spark and Cassandra
To Scale Real-Time Auto Loan
Decisioning At Capital One
Fredrick Crable
Director Software Development
Capital One

30. • Capital One is the #1 bank in the US for loans
• Every loan request needs to be validated
• Legacy depending on 3rd party DSL
• Chose Akka after investigation

33. End result: a decision performance which runs
faster on smaller hardware.
Very happy with Akka (but I suspect they regret
being “too early” in the game)

34. Islands in the Stream: Integrating
Akka Streams and Akka Actors
Colin Breck
Tesla

35. Gather measurements from
a set of wind turbines

36. Attempt with actors, one actor per turbine
• Each actor sends a measure to the DB.
• Actor gets an internal counter to 1000.
• Actor gets a period flush with a scheduler.
• Actor gets some state to see if we just flushed.
• Actor gets some rate limiting mechanism.
Actor has become quite wordy and bug-prone.

37. Akka Streams to the rescue!
getMeasurementSource
.groupWithin(1000, 1.second)
.throttle(50, 1.second, 50, ThrottleMode.shaping)
.mapAsync(1)(seq => dbActor ? newMeasures(seq) …)

38. Source.queue
Source.actorRef
mapAsync + ask
Sink.actorRefWithAck
Bringing streams & actors together
• Streams can be encapsulated within actors
• Actor - Streams communication:

39. Simulating A LOT OF wind turbines
with actor sharding

40. Final demo

41. http://blog.colinbreck.com

42. Speeding up innovation at Verizon
(part of IBM’s talk)
Keyur Shah
Associate Fellow
Verizon

43. • Two years ago: massive monolithic app - 2+
months to rollout new features
• Set up a small team and investigated the
possible platforms.
• They picked up the Reactive platform,
completely microservices driven.

47. They liked it so much that they want to have
all groups in Verizon to adpot it.
Next on the roadmap, try to use all this data
better. Looking at IBM Watson.

48. 8 Akka Anti-Patterns you better be aware of
Manuel Bernhardt
Consultant

49. 1. Using mutable state
• use immutable messages
• use queries like ask for state inquiries
• use pipeTo pattern -- including pipeTo self
instead…

50. 2. Flat actor hierarchies.
If we don't have a hierarchy, we're missing the
point: the most important thing is the failure
tolerance part. No hierarchy == no failure
handling.

51. Behaviour of catch blocks
in some random study

52. 3. Too many ActorSystems

53. 1. string concatenation
2. non-async logging
3. not turning debug logging off in prod system
4. logging to files (we should stop doing that)
4. Wrong logging
a. use Akka's logging facility
b. carefully consider the logback appenders, with async variants
c. use placeholders: log.debug("{} hell", callback)
d. use a logging aggregation like logstash, don’t log to files

54. 4. Lose touch with your hardware

55. We use a lot of abstractions.
VM, then docker, then JVM, then Akka.

56. a. know your hardware (and configure
accordingly)
b. beware of virtualization
c. run load tests on the same hardware of
the production system (!!!)

57. 6. Blocking.
a. use a dedicated dispatcher if you really
must block.
b. if you need to wait, instead of
Thread.sleep, use a context scheduler.

58. 7. Using Akka Remoting directly.

59. Use Akka cluster instead,
with membership service.

60. 8. Using Java serialization.
Bad performance!
Use a proper binary format from the start (avro,
thrift, kryo), because once you are in production
it's already too late.

61. Reactive party

64. Thanks!