ARCHITECTURAL PRINCIPLES FOR CLOUD SOFTWARE

1. ARCHITECTURAL PRINCIPLES
FOR CLOUD SOFTWARE

2. ABOUT ME
SOFTWARE ENGINEER SPECIALIZING IN FULL-STACK (FRONT-END /
BACK-END) DEVELOPMENT. HAVE MORE THAN 3 YEARS OF PRACTICAL
EXPERIENCE WITH ALL STAGES OF THE DEVELOPMENT CYCLE FOR
DYNAMIC WEB PROJECTS.

3. AGENDA
• MONOLITH VS. MICROSERVICES
• EVENT-DRIVEN MICROSERVICES
• WHAT IS CQRS ?
• AWS REVIEW

4. LET’S IMAGINE YOU ARE BUILDING A LARGE,
COMPLEX APPLICATION,
E.G. AN ONLINE STORE

5. SUCCESSFUL SOFTWARE DEVELOPMENT
Architecture
Process Organization
small,
autonomous teams
Agile
CI/CD,
…

6. ARCHITECTURE

8. B

9. BUT SUCCESSFUL
APPLICATIONS KEEP
GROWING…

10. MONOLITHIC ARCHITECTURE
Architecture
Process Organization
small,
autonomous teams
Agile
CI/CD,
…

11. APPLY FUNCTIONAL DECOMPOSITION

13. MICROSERVICE ARCHITECTURE
Architecture
Process Organization
small,
autonomous teams
Agile
CI/CD,
…

14. DRAWBACKS
COMPLEXITY

15. DRAWBACKS
❖COMPLEXITY OF DEVELOPING A DISTRIBUTED SYSTEM
❖COMPLEXITY OF TESTING A DISTRIBUTED SYSTEM
❖COMPLEXITY OF DEPLOYING AND OPERATING A DISTRIBUTED SYSTEM
❖MANAGING THE DEVELOPMENT AND DEPLOYMENT OF FEATURES THAT
SPAN MULTIPLE SERVICES

16. DATA MANAGEMENT PATTERNS

18. LOOSE COUPLING BUT MORE COMPLEX AND …

19. WE ALL KNOW ACTIVE RECORD:

20. EVENT SOURCING IS DIFFERENT
• DON’T SAVE THE CURRENT STATE OF THE OBJECTS
• SAVE THE EVENTS THAT LEAD TO THE CURRENT STATE
• EVENT: SOMETHING THAT HAPPENED IN THE PAST.
E.G.

21. THE ADVANTAGES OF EVENT SOURCING
• COMPLETE LOG OF EVERY STATE CHANGE EVER
• VERY GOOD PERFORMANCE CHARACTERISTICS
• NO MORE MAPPING OBJECTS TO TABLES

22. HOW DOES EVENT SOURCING WORK?

23. SAVING OBJECTS
• CREATE AN EVENT FOR EVERY STATE CHANGE OF THE OBJECT:
• PERSIST THIS STREAM OF EVENTS, PRESERVING EVENT ORDER

24. RESTORING OBJECTS
• SUBSEQUENTLY APPLY THE EVENTS FROM THE RESPECTIVE EVENTSTREAM TO A "BLANK" OBJECT
INSTANCE

25. RESTORING OBJECTS

26. RESTORING OBJECTS

27. RESTORING OBJECTS

28. RESTORING OBJECTS

29. RESTORING OBJECTS

30. DELETING OBJECTS
HOW DO YOU DELETE AN OBJECT?
An event undoing something that happened in the past.

31. CQRS
WTF?

32. CQRS
WE CAN CLEARLY SEE THE SEPARATION BETWEEN
WRITING PARTS AND READING ONES:
THE USER DOES A MODIFICATION ON HIS PAGE,
RESULTING IN A COMMAND BEING EXECUTED.
ONCE THIS COMMAND IS FULLY HANDLED, AN
EVENT IS DISPATCHED TO SIGNAL A MODIFICATION.

33. COMMANDS
• A COMMAND TELLS OUR APPLICATION TO DO SOMETHING. ITS NAME ALWAYS USES
THE INDICATIVE TENSE,
LIKE TERMINATEBUSINESSJOB OR SENDFORGOTTENPASSWORDEMAIL. IT’S VERY
IMPORTANT NOT TO CONFINE THESE NAMES TO CREATE, CHANGE, DELETE… AND TO
REALLY FOCUS ON THE USE CASES.
• A COMMAND CAPTURES THE INTENT OF THE USER. NO CONTENT IN THE RESPONSE IS
RETURNED BY THE SERVER, ONLY QUERIES ARE IN CHARGE OF RETRIEVING DATA.

34. QUERIES
• USING DIFFERENT DATA STORES IN OUR APPLICATION FOR THE COMMAND AND
QUERY PARTS SEEMS TO BE A VERY INTERESTING IDEA. WE COULD CREATE A USER
INTERFACE ORIENTED DATABASE, WHICH WOULD REFLECT WHAT WE NEED TO
DISPLAY TO OUR USER. WE WOULD GAIN IN BOTH PERFORMANCE AND SPEED.
THEREFORE, IT WOULD BE MORE THOUGHTFUL TO USE A DATABASE THAT CAN READ
OUR QUERIES FASTLY.
• IF WE SEPARATE OUR DATA SOURCES, HOW CAN WE STILL MAKE THEM
SYNCHRONIZED? INDEED, OUR “READ” DATA STORE IS NOT SUPPOSED TO BE AWARE
THAT A COMMAND HAS BEEN SENT! THIS IS WHERE EVENTS COME INTO PLAY.

35. EVENTS
• AN EVENT IS A NOTIFICATION OF SOMETHING THAT HAS HAPPENED. LIKE A
COMMAND, AN EVENT MUST RESPECT A RULE REGARDING NAMES. INDEED, THE
NAME OF AN EVENT ALWAYS NEEDS TO BE IN THE PAST TENSE, BECAUSE WE NEED TO
NOTIFY OTHER PARTIES LISTENING ON OUR EVENT THAT A COMMAND HAS BEEN
COMPLETED.
• FOR INSTANCE, USERREGISTERED IS A VALID EVENT NAME.
EVENTS ARE PROCESSED BY ONE OR MORE CONSUMERS. THOSE CONSUMERS ARE IN CHARGE OF KEEPING
THINGS SYNCHRONIZED IN OUR QUERY STORE.
• VERY MUCH LIKE COMMANDS, EVENTS ARE MESSAGES. THE DIFFERENCE WITH A
COMMAND IS SUMMED UP HERE: A COMMAND IS CHARACTERIZED BY AN ACTION
THAT MUST HAPPEN, AND AN EVENT BY SOMETHING THAT HAS HAPPENED.

36. PROS
• SCALABILITY : THE NUMBER OF READS BEING FAR HIGHER THAN THE NUMBER
OF MODIFICATIONS INSIDE AN APPLICATION, APPLYING THE CQRS PATTERN
ALLOWS TO FOCUS INDEPENDENTLY ON BOTH CONCERNS. ONE MAIN
ADVANTAGE OF THIS SEPARATION IS SCALABILITY: WE CAN SCALE OUR
READING PART DIFFERENTLY FROM OUR WRITING PART (ALLOCATE MORE
RESOURCES, DIFFERENT TYPES OF DATABASE).
• FLEXIBILITY : IT’S EASY TO UPDATE OR ADD ON THE READING SIDE, WITHOUT
CHANGING ANYTHING ON THE WRITING SIDE. DATA CONSISTENCY IS
THEREFORE NOT ALTERED.

37. CQRS
• CQRS IS A SIMPLE PATTERN, WHICH ENABLES FANTASTIC POSSIBILITIES. IT
CONSISTS IN SEPARATING THE READING PART FROM THE WRITING PART,
WITH QUERIES AND COMMANDS.
• MANY ADVANTAGES RESULT FROM ITS USAGE, ESPECIALLY IN TERM OF
FLEXIBILITY AND SCALING.
• EVENT SOURCING COMPLETES THE CQRS PATTERN BY SAVING THE HISTORY
THAT DETERMINES THE CURRENT STATE OF OUR APPLICATION. THIS IS VERY
USEFUL IN DOMAINS LIKE ACCOUNTING, BECAUSE YOU GET IN YOUR DATABASE
A SERIES OF EVENTS (LIKE FINANCIAL TRANSACTIONS FOR INSTANCE) THAT
CANNOT BE MODIFIED OR DELETED.

38. SUMMARY
• WHAT’S INTERESTING WITH CQRS IS EVENT SOURCING. IT CAN BE USED WITHOUT
APPLICATION OF THE CQRS PATTERN, BUT IF WE USE CQRS, EVENT
SOURCING APPEARS LIKE A MUST-HAVE.
• EVENT SOURCING CONSISTS IN SAVING EVERY EVENT THAT IS OCCURRING, IN A
DATABASE, AND THEREFORE HAVING A BACK-UP OF FACTS. IN A DESIGN
OF EVENT SOURCING, YOU CANNOT DELETE OR MODIFY EVENTS, YOU CAN ONLY
ADD MORE. THIS IS BENEFICIAL FOR OUR BUSINESS AND OUR INFORMATION
SYSTEM, BECAUSE WE CAN KNOW AT A SPECIFIC TIME WHAT IS THE STATUS OF A
COMMAND, A USER OR SOMETHING ELSE. ALSO, SAVING EVENTS ALLOWS US TO
REBUILD THE SERIES OF EVENTS AND GAIN TIME IN ANALYSIS.
• CQRS IS A SIMPLE PATTERN, WHICH ENABLES FANTASTIC POSSIBILITIES. IT CONSISTS
IN SEPARATING THE READING PART FROM THE WRITING PART,
WITH QUERIES AND COMMANDS.

39. AWS
• IN REACTIVE, MESSAGE-DRIVEN APPLICATIONS IT IS CRUCIAL TO DECOUPLE PRODUCERS AND
CONSUMERS OF MESSAGES. COMBINING PUBLISH/SUBSCRIBE (PUB/SUB) AND QUEUEING
COMPONENTS WE ARE ABLE TO BUILD RESILIENT, SCALABLE AND FAULT-TOLERANT APPLICATION
ARCHITECTURES. AWS OFFERS A VARIETY OF COMPONENTS WHICH IMPLEMENT PUB/SUB OR
QUEUEING.
• TWO SIMPLE BUT POWERFUL COMPONENTS FOR EVENT AND MESSAGE PROCESSING ON AWS:
THE SIMPLE NOTIFICATION SERVICE(SNS) AND THE SIMPLE QUEUE SERVICE (SQS).

40. OUR GOAL
• THE GOAL IS TO DEVELOP AN EVENT PIPELINE WHICH SENDS A MESSAGE TO A SLACK CHANNEL WHENEVER
SOMEONE UPLOADS A PICTURE TO AN S3 BUCKET. FOR DEMONSTRATION PURPOSES WE WILL ALSO STORE
THE EVENTS IN A QUEUE FOR ASYNCHRONOUS PROCESSING. THE ARCHITECTURE INVOLVES S3 EVENT
NOTIFICATIONS, AN SNS TOPIC, AN SQS QUEUE, AND A LAMBDA FUNCTION SENDING A MESSAGE TO THE
SLACK CHANNEL. HERE IS AN ANIMATION OF THE FINAL RESULT.

41. AWS

42. ARCHITECTURE
• LET'S LOOK AT THE HIGH LEVEL ARCHITECTURE. WHEN A CLIENT UPLOADS AN IMAGE TO THE
CONFIGURED S3 BUCKET, AN S3 EVENT NOTIFICATION WILL FIRE TOWARDS SNS, PUBLISHING
THE EVENT INSIDE THE RESPECTIVE TOPIC. THERE WILL BE TWO SUBSCRIBERS FOR THAT TOPIC: AN
SQS QUEUE AND A LAMBDA FUNCTION.
• THE SQS QUEUE STORES THE EVENT FOR ASYNCHRONOUS PROCESSING, (E.G. THUMBNAIL
GENERATION OR IMAGE CLASSIFICATION). THE LAMBDA FUNCTION PARSES THE EVENT AND
SENDS A NOTIFICATION MESSAGE TO A SLACK CHANNEL. WITHIN THE SCOPE OF THIS WE ARE
NOT GOING TO DISCUSS THE ASYNCHRONOUS PROCESSING PART. DUE TO THE DECOUPLING
OF PUBLISHING AND SUBSCRIBING WITH SNS WE ARE FREE TO ADD MORE CONSUMERS FOR
THE EVENTS LATER.

43. ARCHITECTURE
• LET'S LOOK AT THE INDIVIDUAL COMPONENTS IN DETAIL. S3 ORGANIZES OBJECTS IN BUCKETS.
WITHIN A BUCKET YOU CAN REFERENCE INDIVIDUAL OBJECTS BY KEY. UPLOADING A FILE TO
S3 CAN EITHER BE DONE VIA THE AWS CONSOLE, THE AWS CLI, OR DIRECTLY THROUGH THE
S3 API.
• S3 ALLOWS TO CONFIGURE EVENT NOTIFICATIONS. EVENTS CAN BE CREATED BASED ON
OBJECT CREATION OR DELETION. YOU CAN CHOOSE TO EITHER SEND THE EVENT TOWARDS AN
SNS TOPIC, AN SQS QUEUE, OR A LAMBDA FUNCTION.
• IN OUR CASE WE ARE GOING TO SEND THE EVENTS TO SNS AND THEN ALLOW INTERESTED
APPLICATIONS TO SUBSCRIBE. INSTEAD OF SENDING EVENTS DIRECTLY TO ALL PARTIES, BY USING
SNS AS AN INTERMEDIATE BROKER WE DECOUPLE PUBLISHING AND SUBSCRIPTION.

44. ARCHITECTURE
• SNS IS A SIMPLE PUB/SUB SERVICE WHICH ORGANIZES AROUND TOPICS. A TOPIC GROUPS
TOGETHER MESSAGES OF THE SAME TYPE WHICH MIGHT BE OF INTEREST TO A SET OF
SUBSCRIBERS. IN CASE OF A NEW MESSAGE BEING PUBLISHED TO A TOPIC, SNS WILL NOTIFY
ALL SUBSCRIBERS. YOU CAN CONFIGURE DELIVERY POLICIES INCLUDING CONFIGURATION OF
MAXIMUM RECEIVE RATES AND RETRY DELAYS.
• THE GOAL IS TO SEND A SLACK MESSAGE ON OBJECT CREATION WITHIN OUR S3 BUCKET. WE
ACHIEVE THAT BY SUBSCRIBING A LAMBDA FUNCTION TO THE SNS TOPIC. ON INVOCATION
THE LAMBDA FUNCTION WILL PARSE AND INSPECT THE EVENT NOTIFICATION, EXTRACT
RELEVANT INFORMATION, AND FORWARD IT TO A PRECONFIGURED SLACK WEBHOOK.

45. Q&A