We’ll consider a process of building specific Java backend application that must survive a high load.
During the talk we’ll analyse requirements, pick up a technical stack and then while discussing specific functionality go through typical major painful points that specifically Java developers are not taking into account that eventually result in inability to run such application at large scale, breach response times, crash via OOM and some others. Following the discussion about potential architectural flaws, we’ll also discuss recommended approaches, techniques and solutions for tackling those.
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
JEEConf 2019 | Let’s build a Java backend designed for a high load
1. Let’s build a Java backend designed for
a high load
Alex Moskvin
CTO@Plexteq
2. About myself
• CTO@Plexteq OÜ
• Ph.D in information technology area
• Interests
• Software architecture
• High loaded systems
• Everything under the hood
• AI/ML + BigData
• Knowledge sharing ;)
• Follow me
• https://twitter.com/amoskvin
• https://www.facebook.com/moskvin.aleksey
2
10. Problem :: key aspects
1. Interaction with
remote/external
services
2. Processing data from
remote/external
services
3. Storing data
4. Handling interaction
with clients
10
20. Distributed services :: handle errors!
Issue #2:
No error handling
Solution
1. Handle them all
2. Check response code
• i.e. 429 is a requirement to stop sending requests
20
21. Distributed services :: fail fast
Issue #3:
Code relies on high availability of external service
Solutions:
1. Fail-fast
21
23. Distributed services :: circuit breaker
Stability pattern used when calling remote functions
23
24. Distributed services :: Circuit breaker
CLOSED OPEN HALF OPEN
If state = OPEN && grace period passed -> retry
If request succeeded -> CLOSED
Otherwise -> OPEN
Service is OK Service is FAILING
24
30. Processing data :: deserialization
Issue #1:
Deserialization is expensive with large payloads
30
31. Processing data :: deserialization
Issue #1:
Deserialization is expensive with large payloads
Consequences:
1. Heap saturation -> potential OOM
2. GC pauses -> high latencies
3. Deserialization is sequential -> thread is not
doing anything useful until deserialization
completes
31
48. Storing data
• No GC involved
• Low heap usage (10x time smaller)
• Map is shared between multiple JVM processes
• Map could be replicated across multiple nodes
(commercial feature)
48
51. Storing data
Relational storage. Solution:
1. Avoid pessimistic locking
2. Transaction isolation > Read Committed is a no-go
3. Scale up!
4. Connection pool implementation matters (HikariCP)
5. Connection pool sizes on app side and on DB size must
correlate
6. Use indices wisely
7. Know your execution plans!
51
63. Handling client requests
Pros:
1. System is more predictable
• Target resource (i.e. MongoDB) load is managed and constrained
with a thread pool (a kind of DOS protection)
• Thread allocation is managed and constrained
63