The document discusses benchmarking methods for Magento 2, highlighting the variability in results based on different caching configurations and test setups. It summarizes tests using JMeter with various configurations, such as Redis and Varnish caching, and presents performance metrics like average response times and their scalability issues. The findings suggest that while caching is beneficial, Varnish is the fastest option, and there are concerns about the efficiency of Redis in multistore environments.

1. Magento 2 benchmarks
Daniel Genis

2. A note about benchmarks (in general)
Not representative
Easy to make mistakes
Different methods
Different ways of measuring
Cold/Hot caches
Many subtle choices
Complexity

3. The benchmark test website
Sample data magento2
not many products
no plugins
about 3000 unique frontend URLs
Benchmarking frontend only
Zero cache invalidations

4. The benchmark tools
JMeter test plan, 20 threads doing:
Request a frontend page + all static content
upon completion wait 2 seconds
Request a different frontend page + static content
Each thread has a pre-generated list of urls
Each thread makes 50 requests per default
A warming run was done before actual benchmark

5. The test server
Hypernode - Magento Grow
2 cpu cores
2 gb ram
SSD storage
Small test server
Easier to push to it's limits
And to make a point!

6. Clean install - no redis, no varnish
20 threads 50 req / thread

7. With redis caching, no varnish
20 threads 50 req / thread

8. Redis caching 10 threads
10 threads 100 req / thread

9. With varnish caching
20 threads 50 req / thread

10. Statistical summary
clean install redis cache redis 10thr varnish cache
Average 412 ms 403 ms 278 ms 125 ms
Median 378 ms 372 ms 263 ms 116 ms
95th
percentile
727 ms 710 ms 423 ms 187 ms

11. Observations
Caching is easy to setup
Varnish cache in built
Redis cache in built
No plugins needed
These benchmarks were very bandwidth sensitive
Static content!
html: ~10kb
total incl static ~1mb

12. Observations
Varnish naturally is the fastest setup
Redis caching vs On disk caching (default)
frontend wise, no measurable improvement
local storage is cached in ram (hence no difference)
NFS ?
Scalability ?

13. Magento 2 docs about redis
Cache entries are stored in 'core_cache_tag' table
On multistore environment this table can grow very large
SELECTS are fine
but UPDATE/INSERTS become more work (scalability issue)
Redis is more efficient (than memcached or default cache)
For various query/update scenarios
Has high availability (i.e. master/slave replication)

14. Bedankt!
Blijf op de hoogte van de laatste ontwikkelingen op
hosting-gebied. Volg ons op twitter: @Byte_NL