Meetic back-end redesign in microservice by Etienne Broutin @meetictech during @meetup_ms_paris

1. Meetic back-end redesign
in micro-services
Etienne Broutin
Software architect
@meetictech
1

2. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Who am I ?
2
Joined Meetic in 2014
Interested by their volumes,
functional richness and
experience
Etienne
Broutin
Software Architect

3. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
3
Started in 2001
•
Active in 16 countries
•
Dating leader in Europe
•
Millions of Monthly Active Users
•
150 people in IT teams

4. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
2013 - refactoring plan
Unmaintainable code
Duplicates
Untestable
Need elders validation
Unable to monitor
Continue on failure policy
Based on global log volume
4

5. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
2013 - refactoring plan
55
Private webservices
Webservices
Backoffices
Mobile Web
WAP
Desktop
Cronjobs
…
Exposition Layer

6. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Started
6
So cute in the
beginning !

7. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
New issues
Duplicated business logic
Merge conflicts
Longer tests (40min)
Refactorings started to become more and more
difficult
7

8. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
8
We were building a new monolith !
Only 15% of business features
Only 10 developers, target 40

9. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Micro-service pros
● Simpler design phase
● Manage refactoring impacts
● Faster feedbacks by software factory
● Faster deployments
● Real ownership by developers
9

10. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Micro-service cons
● Implementation of interfaces
● Response time
● CPU and network usage
● Time to setup a new application
● Can also become hard to understand
10

11. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
New target
11
Exposition Layer
Event bus
Consumers
Micro-services
BackOffices
Webservices

12. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
2 years later
12
Micro-service design
Dependencies
Tea
m
Production

13. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Micro-service design
13

14. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
How to split code
Functional splitting
Built around a concept
14
Pictures Right

15. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
What size
15
30 micro-services
9 avg route count
7k avg NLOC
4 avg relational tables
to understand
a micro-service
1 to 2
days

16. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Testability
16
Build in 3minSimpler functional
tests

17. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Modelization
17
POST /photos/{accountId}
GET /photos/{accountId}/{photoId}
PUT /photos/{accountId}/{photoId}
DELETE /photos/{accountId}/{photoId}
POST /boost/{boostId}/increment
REST approach Commands

18. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Data Isolation
18
Strong and difficult choice But
• brings scalability
• brings visibility
• data consistency
• enables caching
• manage data migration

19. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Limit to data isolation
19
Build a denormalized
database
OLTP
Reactive (event bus)
Batch processing
BI

20. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
servicesHow to deal with
dependencies
20

21. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Product has a new idea
21

22. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
In practice
22
2/3 of our micro-services
do not depend
on another micro-service
It took us several
refactorings for that

23. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Defining perimeter is the key
23
Messages Authentication
nickname
business model
account status
Photo Announce Moderation
Photo +
Moderation
Announce +
Moderation

24. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Event bus to manage depencies
24
Event bus
Consumer
Consumer
Profile Mailing Optin

25. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Pattern 1 : pull data
25
get photos
does have a
photo itself?can see photos?
Exposition
Layer
Rights Photo
- performance
- reusability
+ hides complexity for clients
+ fast change of business rules

26. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Pattern 2 : push data from client
26
send message
Exposition Layer
Rights
Message
conversation
already started ?
can start
a new conversation ?
+ mutualize data fetching - harder to change if multiple clients

27. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Pattern 3 : store data
27
Event bus
read message
Exposition
Layer
Messageget
visible
messages
Blacklist
blacklist
added
hide
messages
+ performance
+ reusable
- long to implement

28. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Other tips
28
Batch calls
GET /photos/{accountId}/14,15,21
Parallelize calls
using Guzzle promise

29. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Working together
29

30. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Context
30
15 deployments
per day
Several independent agile teams
No strong synchronization
Continuous deployment

31. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
How do you manage compatibility?
31
No versioning
• used internally
• less maintenance
• interface backward
compatible

32. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Automated environment
32
Scripts with docker compose

33. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
33
Can I use any
framework ?
Any
langage ?
Any database ?

34. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Common interface
34
• REST + JSON
• Naming conventions
• Security
• Common build
• Common deployment chain

35. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Don’t waste time
we need a common basis
35
Micro-services chassis
• Security
• Logging
• HTTP Interface
• DB & cache components

36. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Share knowledge with non-tech team
36
Talkative names
Visibility for project management
profile will be
updated soon
changes on
message are
ready
thanks

37. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Documentation
37
Inventory
<<<
Call graph generated
from Elastic Search
GET
/profile/{accountId}
Profile
Exposition Layer
Right
Backoffice 4%
85%
11%

38. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Ownership
38
More initiatives
More refactorings
More upgrades
More monitoring

39. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Production
39

40. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Hosting concerns
40
It will overload the network !
It will take time to configure !
We will need much more
servers !

41. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Figures
41
2B calls / day
25ms response time
100 servers
Who calls micro-services ?
Exposition LayerEvent
Bus
Micro-services

42. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Networking
42
• Every micro-service deployed on each server
Have not experienced need for :
• Service isolation
• Network constraints

43. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Symfony bootstrap overhead
43
8 ms limited by application size
17 ms
25ms
Cost : 25%

44. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Monitoring
44
Alerting
Elastic Search + Logstash + Kibana
• Easy to locate errors
• Performance analysis
GET /profile/{accountId} 11ms
100%
POST /profile/
42ms 99.992%
...
Monitoring route for every dependency
✓
✓

45. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Design for failures
45
Objective 99.999%
Internally : analyse logs
Externally : implement fallback behavior
Exposition Layer
99.9% 98.3% 99.2%
Reliability < 97.4%

46. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Conclusion
46

47. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Re-usability
47
Re-used micro-services
from acquisition fusion
Used for several products

48. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Huge opportunity for data
48
● Best database for each
● Scalability

49. @matterstech
meetup.matters.tech
Meetic back-end
redesign in micro-
services
Feedback of that choice
49
For Meetic, this architecture was a good choice :
• Symfony helps standardization
• Can scale teams
• Clear implementation of all features
• Answer business needs

50. Any questions ?
@matterstech
https://meetup.matters.tech
50