Java/Hybris performance monitoring and optimization

Mykhailo Drach
Andrii Davydenko

CONFIDENTIAL | © 2020 EPAM Systems, Inc.
Presenters
Mykhailo Drach
Hybris Software Engineer @EPAM
Andrii Davydenko
Senior Performance Analyst @EPAM

2
1
3
Performance
metrics
This part introduces performance metrics. As defining
performance criteria and desirable parameters is the
starting point of performance improvement process.
Performance
monitoring
This part tells us about continuous monitoring of
enterprise Java applications.
Performance
improvement
process
This part is about optimization and tuning process of Java
applications.
Agenda

Response time Responsiveness Load Capacity Latency Throughput Efficiency Scalability
Response time
Response time is amount of time system takes to react to user
request
Response time metrics:
• Peak value
• Average, percentile value
• Error rate
Reference: Patterns of EAA by Martin Fowler
Percentile calculation

Responsiveness
Responsiveness is about how quickly the system loads part of
response data. If your system waits during the whole request, then your
responsiveness and response time are the same. The earlier user can
interact with your site the better responsiveness it has.
Poor responsiveness Good responsiveness
Examples:

Load
Load is a statement of how much stress a system is under, which
might be measured in how many clients are currently connected to it.
The load is usually a context for some other measurement, such as a
response time. Thus, you may say that the response time for some
request is 0.5 seconds with 10 users and 2 seconds with 20 users.
Examples:
• 1000 Users are currently connected to the server
• 200 Active user sessions

Response time Responsiveness Load Throughput Capacity Latency Efficiency Scalability
Throughput
Throughput is how much stuff you can do in a given amount of
time. If you’re timing the copying of a file, throughput might be
measured in bytes per second. For enterprise applications a typical
measure is transactions per second (tps), but the problem is that this
depends on the complexity of your transaction.
Examples:
• Server can place 100 orders/min
• DB can commit 100 transaction/s

Capacity
The capacity of a system is an indication of maximum effective
throughput or load. This might be an absolute maximum or a point at
which the performance dips below an acceptable threshold.
Examples:
• Server capacity is 10000 active user sessions
• Server capacity is 2000 active third party clients

Latency
Latency is the minimum time required to get any form of response,
even if the work to be done is nonexistent. It’s usually the big issue in
remote systems. As an application developer, you can decrease latency
impact minimizing remote calls.
Examples:
• Ping to server is 50 ms

Efficiency
Efficiency is performance divided by resources.
Examples:
• A system with maximum throughput in 30 tps on two CPUs is more efficient than a
system that processes 40 tps on four identical CPUs.
• A system with 1000 active user`s sessions and 10 GB Ram is more efficient
than system with 1200 active user`s sessions and 15 GB Ram

Scalability
Scalability is a measure of how adding resources (usually hardware)
affects performance. A scalable system is one that allows you to add
hardware and get a performance improvement.
Scalability types:
• Vertical scalability, or scaling up, means adding more power to a
single server, such as more memory.
• Horizontal scalability, or scaling out, means adding more servers.

Performance monitoring

Roadmap
Performance testing matters. E-Commerce
Preparation fot testing
Toolset
Typical issues

Performance testing matters

Market growth
• 10% increase in 10 years
• >56% of dollar gain made online
• Switching to mobile is growing
• Impact of COVID-19 lockdown

Why performance test
• Cost of performance
issues, reputational risks
• Disaster recovery plan
• Cost optimization
• Building a culture of
performance focused development
• Performance Testing Under CI/CD

Preparation for testing

NFRs & Load model
• Capacity in terms of users/tasks
• Response/processing time, aggregation
type
• Infrastructure setup
• Resource consumption
• Error conditions
• Scalability
• Review PROD logs,
Adobe/Google analytics, business data
• New product: business data, competitors
• Distribution in user types, devices on
client side

Diff
Percentiles
• allow to understand the
distribution
• perfect for automatic
baselining
• better for tuning and
defining a goal for
optimizations
*** A Bell curve describes
simplified distribution
when average and median
response times are equal.
Almost impossible in real
life.

Estimate capacity
Active users
DB CPU
Hits/sec

Performance environment
Best case
• Scaled as PROD
• Integrations in place
• CDN not ignored
• Data restored from PROD with
masked data, refreshed on
regular basis
• New product: generated volume
to match expectation
Worst case
• Development machine hosting
PERF, load generators, updating
Facebook, etc.
• Minimum of data

Toolset

What we can use
Backend
• JMeter, Tsung
• TeraVM, LoadRunner
Frontend
• JMeter+WebDriver
• Sitespeed.io
Analysers
• DevTools
• Lighthouse, HttpWatch
• Wireshark, Fiddler
APMs
• Dynatrace
• NewRelic
• AppDynamics
Profilers
• Jet Profiler
• JProfiler
• Etc.
hac
• Cache
• Thread Dump
• Database

Hybris typical issues

Important notes
• Cart creation, cleanup strategy,
recalculations, merging,
• FlexibleSearch caching, eviction
strategy
• Cluster configuration: tasks
engine, special node groups
• Page fragment caching *
• Table locking *

Cronjob related issues
• Executed on dedicated node group
• Setting up sequential cronjobs when
applicable, plan enough buffer time
• Scheduling at non-peak hours
• Assess impact of 3rd party services
• Prioritize list you have
• Enough data to process during tests

BO <bombs_away> search

Response time and GC issues

Performance improvement

Software improvement types
• Software optimization
• Optimization - identifying and eliminating internal
inefficient designs and implementations
• Software tuning
• establishing the optimal setting for every possible
external configuration parameter

Before starting performance improvement process, we should define
desired performance results.
How to do this:
• Analyze business
• Ask stakeholders
• Find best practices
Defining desired params
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

Application performance can be improved by tuning jvm properties.
Configs to look at:
• Memory size
• GC algorithm
• ThreadPool size
• Caching
Application config tuning
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

Heap size is one of the most important parameters. We have to define best gc
time – performance value.
Parameters : -Xms – initial heap size -Xmx – maximum heap size
Oracle recommends setting –Xms equal to –Xmx to reduce garbage collections
Heap size
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

Thread pools have a minimum and maximum number of threads. The
minimum number of threads is kept around, waiting for tasks to be assigned to
them. The maximum number of threads also serves as a necessary throttle,
preventing too many tasks from executing at once.
ThreadPool size parameters
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

The most used GC algorithm:
• The serial garbage collector
• The throughput collector
• The CMS collector
• The G1 collector
GC algorithm
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

Code optimization is the most important and biggest part of performance
improvement process. It means that you need to redesign and reimplement some
part of code.
Examples:
• SQL Queries
• Concurrency
• Looking for rid operations
• Reducing external calls
Code optimization
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

DB Querying in large systems can be the biggest hotspot. To avoid it we should
carefully investigate each query and tune it in order to improve performance.
What can be improved?
• Joins
• Indexes
• Generated queries analysis
Code optimization: DB Queries
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

In relational databases all data is stored in tables (relations). In most case to get
data we have to join tables.
How to improve joins?
• Indexes
• Views
Code optimization: DB table joins
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

To improve SQL lookup and joins we can use indexes. Index is a special lookup
table that stores data into tree-based form.
Code optimization: DB table indexes
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

Pros:
• It speeds up join and data lookup
• Data lookup – O(log2N)
Cons:
• It slows down Create, Update and Delete operations
• It extends data size
Code optimization: DB table indexes
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

Java 8 Stream API is a convenient way to support a functional approach to
processing collections of objects. In addition it can speedup data processing.
Java 8 Stream API provides possibility to parallelize data processing with quite
simple API.
Code optimization: Prefer Streams over loops
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

If you use ORM in your Java application then it keeps in the session a version
of the entities already persisted, just in case they are modified again before
session was closed. It fills memory and can end with OutOfMemoryError. To
fix it you can either avoid long running ORM sessions or clear
PersistenceContext from time to time.
Code optimization: Long running ORM sessions
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

Memory analyzing is the process of defining main application memory
parameters. It includes leaks finding
Tools for monitoring:
Memory analyzing
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance
Java VisualVm Eclipse memory analyzing tool

Application profiling
Stackify Prefix
JProfiler YourKit
Java Profiling tools
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

Software profiling is a form of dynamic program analysis that measures,
for example, the space (memory) or time complexity of a program, the frequency
and duration of function calls.
Types of profiling:
• Sampling
• Instrumental
Application profiling
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

Adding more hardware resources improves performance. But you should make
code optimization and config tuning before.
Scaling application resources
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance

Desired performance
Defining desired
params
Config
tuning
Code
optimization
Memory
analyzing
Profiling Scaling
Desired
performance
If you finished all previous steps then your
performance should be at desired level.
What to do next to keep it working well?
Next steps:
• Looking for performance issues in code
reviews
• Looking for possible performance issues
implementing new features
• Continuous performance testing

Java/Hybris performance monitoring and optimization

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Java/Hybris performance monitoring and optimization

Similar to Java/Hybris performance monitoring and optimization (20)

Recently uploaded

Recently uploaded (20)

Java/Hybris performance monitoring and optimization