The document discusses event-driven architectures using Spring Cloud and related tools. It covers event sourcing as a historical concept dating back to ship logs, and how Apache Kafka provides an immutable log for event storage. Microservices are discussed as benefiting from event-driven designs due to lack of foreign key constraints. CQRS and materialized views are presented as techniques for generating queries from events. A reference architecture is proposed for a social network application built as microservices that use event sourcing and stream processing to maintain consistency across services.

1. © 2017 Pivotal Software, Inc. All rights reserved.
@kennybastani
Event-driven Architectures with
Spring Cloud
Kenny Bastani
Spring Developer Advocate @ Pivotal
1

2. Kenny Bastani
@kennybastani
Spring Developer Advocate

3. The methods
• Microservices
• Domain-driven Design
• Event sourcing
• CQRS
• Cloud native applications

4. The tools
• Apache Kafka
• Spring Cloud Stream
• Spring for Apache Kafka
• Spring Boot
• Spring Cloud

5. © 2017 Pivotal Software, Inc. All rights reserved.
@kennybastani
The agenda
• Event sourcing, a history
• Apache Kafka and the immutable log
• Event-driven microservices
• Spring Cloud Stream + Kafka
• Reference architecture
5

6. © 2017 Pivotal Software, Inc. All rights reserved.
@kennybastani
Event sourcing
A mystery, a history…
6

7. Origin story

8. Event Sourcing ensures that all changes to application state are
stored as a sequence of events. Not only can we query these events,
we can also use the event log to reconstruct past states.
We can determine the application state at any point in time. We do this
by starting with a blank state and rerunning the events up to a
particular time.
–Martin Fowler

9. Did Martin Fowler invent event sourcing?

10. Edgar Allan Poe (1804-1849)
We appreciate time by events alone. For this reason we
define time (somewhat improperly) as the succession of
events; but the fact itself--that events are our sole means
of appreciating time-- [leads to] the erroneous idea that
events are time--that the more numerous the events, the
longer the time; and the converse.

11. Did Edgar Allan Poe invent event sourcing?
Lol no…

12. Ship
Chip Log
15th-16th Century
Drag
Knots
Ocean
Log Reel
Log-line

13. © 2017 Pivotal Software, Inc. All rights reserved.
@kennybastani
The Poe connection
• Poe was obsessed with the idea of event sourcing for much of his life.
• But in his time, things were commonly measured using a form of event
sourcing, like the knots on the log-line for measuring the speed of
ships.
• Most software is built on metaphors. The problem is, we’ve forgotten
the metaphors have meaning.
13

14. Marginalia, Edgar Allan Poe, 1849

15. Marginalia, Edgar Allan Poe, 1849

16. Ship
Chip Log
15th-16th Century
Events (Objects)
Aggregation takes causal events across space and time and
summarizes it as a single object that’s easier to store in memory
Time
Space

17. Poe was a master of language, metaphor, and
abstraction.
He understood the method of computation through
language before computers existed.

18. “An artist will always contrive to weave his illustrations into the metaphorical form.”
Also, Edgar Allan Poe, 1849

19. All great software is built on metaphors
Desktop
Cloud
Web
Infrastructure
Virtual MachineState
Events
Services
Logs
Kernel
Tablets
Application
Instance
Space
Timestamp
Memory
Containers
Database
Platform
Computer
Refactor
Files
Pages
Documents

20. The metaphors mean something
• There’s an underlying abstraction to our own humanity and it has an
intimate connection to the virtual world we are building
• The desktop computer replaced actual desktops by replacing the
abstract functionality of the desktop
• The virtual machine replaced the server by virtualizing the abstract
functionality of the hardware

21. But ultimately, it’s hard to keep track of
all the layers.

22. Eventually we get lost in abstraction.

23. Serverless
Microservices
NoSQL
SOA
Middleware
All confusing software is built on nonsense
DevOps
ESB

24. I’m not saying microservices, DevOps,
and serverless are nonsense.

25. I’m saying that metaphors make good
ideas more durable to nonsensical
interpretation.

26. Ship
Chip Log
15th-16th Century
Drag
Knots
Ocean
Log Reel
Log-line
LOG!

27. LOG!

29. © 2017 Pivotal Software, Inc. All rights reserved.
@kennybastani
Apache Kafka
The immutable log as the source of truth
29

30. Time-ordered events

31. Store state as a log of events
https://www.confluent.io/blog/messaging-single-source-truth/

32. Ship
Chip Log
15th-16th Century
Drag
Knots
Log Reel
Log-line
It’s the same thing, but way before software

33. Event sourcing isn’t just a good idea.
It’s how computers work.

34. Universal Turing Machine

36. So why aren’t we doing it everywhere?

37. Because storing state as an immutable
log of events for domain data didn’t
make sense in a time B.C.

38. B.C. = Before Cloud
– Andrew Clay Shafer

39. https://www.confluent.io/blog/messaging-single-source-truth/

40. https://www.confluent.io/blog/messaging-single-source-truth/

41. The Immutable Log
https://www.confluent.io/blog/messaging-single-source-truth/

42. So why now? Why are events a big deal?

43. © 2017 Pivotal Software, Inc. All rights reserved.
@kennybastani
Microservices!
Why do we need event-driven microservices?
First, let’s build an event sourced app
43

44. © 2017 Pivotal Software, Inc. All rights reserved.
@kennybastani
Spring Boot
A JVM micro-framework for building microservices
44

45. What is Spring Boot?

46. @kennybastani
spring boot
Spring Initializr for boostraping your applications
supports rapid development of
production-ready applications
and services

47. Apache Kafka & Spring
• Spring for Apache Kafka
– Integrates Kafka Streams API
– Integrates Kafka Messaging API
• Spring Cloud Stream
– Swappable binders for different message brokers
– Supports Kafka, RabbitMQ and more

48. Why do we need event-driven
microservices?

49. There are no foreign key constraints
between microservices.

50. Domain Graph

51. Domain
Relationships

52. No foreign key constraints across microservices
Foreign
Key
Relationships

53. Distributed joins are slow (reads)

54. Distributed transactions are brittle (writes)

55. Microservice joins
• Microservices are organized
using domain-driven design
• Microservices cannot own
domain data from two different
bounded contexts
• Complex joins (reads) are
inevitable
• Avoid distributed transactions
(write)

56. © 2017 Pivotal Software, Inc. All rights reserved.
@kennybastani
CQRS
Designing REST APIs to generate domain events
56

57. What is CQRS?
CQRS (Command Query Responsibility Segregation) is a
design pattern for turning events into materialized views

58. @spring_io
#springio17
Commands and Queries

59. COMMANDS
GENERATE
EVENTS

60. EVENTS
ARE BROADCASTED
TO OTHER SERVICES

61. EVENT LISTENERS
BUILD QUERY
MODELS

62. API GATEWAYS HIDE THE
IMPLEMENTATION DETAILS

63. © 2017 Pivotal Software, Inc. All rights reserved.
@kennybastani
Reference architecture
Creating a social network as microservices
63

64. How do you do friend of a friend
queries with microservices?

65. User User
UserUser User
Friend
Friend Friend
FriendFriend

66. User User
UserUser User
Friend
Friend Friend
FriendFriend
Day 1 of building microservices
Day 2 of building microservices…

67. Materialized views

68. Create aggregate views from event data

70. • Manage the storage of domain data that it owns.
• Produce the API contract for the domain data that it owns.
• Produce events when the state of any domain data changes.
• Maintain relationship integrity to domain data owned by other services.

71. • Subscribe to domain events emitted by separate domain services.
• Maintain an ordered immutable event log for events it receives.
• Create connected query projections of distributed domain data.
• Provide performant read-access to complex views of domain data.

74. Transaction logs

75. Event Sourcing
Aggregates can be used to generate the consistent state of any object
It provides an audit trail that can be replayed to generate the state of an object from
any point in time
It provides the many inputs necessary for analyzing data using event stream
processing
It enables the use of compensating transactions to rollback events leading to an
inconsistent application state

76. Event Types

77. Event Log

78. Event logs are transaction logs
Use event sourcing in your microservices to ensure consistency of your
distributed system
Don’t store state with microservices, store events
Use event stream processing to observe the active usage of your cloud
native application in real-time

79. © 2017 Pivotal Software, Inc. All rights reserved.
@kennybastani
Thanks!
Q/A
79
https://github.com/kbastani/event-sourcing-microservices-basics
kennybastani.com