1. ppbench
A Visualizing Network
Benchmark for Microservices
Nane Kratzke and Peter-Christian Quint
1
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems

2. The next 15 to 20 minutes are about ...
• The cloud-native application stack and microservices, containers
and container clusters (Docker Swarm, Mesos, Kubernetes, etc.)
• Possible technical and performance impacts of above mentioned
approaches
• A benchmarking tool for figuring out performance impacts in the
upfront of a microservice design.
• Some evaluation results of ppbench.
• Some performance insights you might not know about Ruby, Java,
Go, Dart, Docker (container) and overlay networks like Weave!
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
2

3. Microservices
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
3
„The microservice architectural style is an approach to developing a
single application as a suite of small services, each running in its
own process and communicating with lightweight mechnisms, often
an HTTP resource API.“
Martin Fowler

4. The Cloud-Native Stack (ClouNS)
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
4
(1) Different Programming Languages
(2) Overlay Networks
(3) Container
(4) VM Types

5. However, there is missing a tool ....
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
5
to figure out possible impacts of
z different programming languages
z overlay networks
z additional container layer
z different VM types
z ...
in the upfront of a microservice design.

6. Reference Experiment
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
6
Experiment to measure reference network performance of REST-like
HTTP-based communication protocols.
Ping and pong services are
implemented in different
TIOBE TOP 50 programming
languages.

7. And we get visualizations ...
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
7
Data Transfer Rates
(bigger is better)
Message Size (kB)
TransferRate(MB/sec)
0 kB 50 kB 150 kB 250 kB 350 kB 450 kB
0MB/sec16MB/sec32MB/sec48MB/sec64MB/sec80MB/sec
● bare on m3.2xlarge For identifying
- problematic areas
- skewness of distributions
- across a wide range of
message sizes
In that case. Dart‘s http
standard libraries seem to
have problems around
message sizes bigger than
three times TCP standard
receive window (dotted
lines).
Skewness is
shifted to bottom
(not preferable).
Skewness is
shifted to top
(preferable).

8. Impact of Containers
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
8
Experiment to measure container impact on network performance of
REST-like HTTP-based communication protocols.

9. Impact of Software Defined Networks
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
9
Experiment to measure additional SDVN impact on network
performance of REST-like HTTP-based communication protocols.
weave

10. Language Impact
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
10
Data Transfer Rates
(bigger is better)
Message Size (kB)
TransferRate(MB/sec)
0 kB 50 kB 150 kB 250 kB 350 kB 450 kB
0MB/sec22MB/sec44MB/sec66MB/sec88MB/sec
●
●
●
●
Go (Exp. P1) on m3.2xlarge
Java (Exp. P2) on m3.2xlarge
Ruby (Exp. P3) on m3.2xlarge
Dart (Exp. P4) on m3.2xlarge
Requests per second in relative comparison
(bigger is better)
Message Size (kB)
Ratio(%)
0kB 50kB 150kB 250kB 350kB 450kB
0%20%40%60%80%110%140%170%200%
●
●
●
●
Reference: Go (Exp. P1) on m3.2xlarge
Java (Exp. P2) on m3.2xlarge
Ruby (Exp. P3) on m3.2xlarge
Dart (Exp. P4) on m3.2xlarge
(1) Different Programming Languages
(1) Language impact should not be undererstimated
(2) Do not overestimate Go.

11. Overlay Network Impact
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
11
Data transfer in relative comparison
(bigger is better)
Message Size (kB)
Ratio(%)
0kB 50kB 150kB 250kB 350kB 450kB
20%50%80%120%
●
●
●
Reference (Exp. S1, S3) on m3.large
Go, SDN deployed (Exp. S2 compared with S1) on m3.large
Ruby, SDN deployed (Exp. S4 compared with S3) on m3.large
(2) Overlay Networks
(1) OK, SDN decreases performance. Not astonishing.
(2) But, SDN can sometimes improve performance!
weave

12. Container Impact
is likely to be hardly measurable in real world scenarios
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
12
Round−trip latency in relative comparison
(smaller is better)
Message Size (kB)
Ratio(%)
0kB 50kB 150kB 250kB 350kB 450kB
90%110%130%
●
●
●
Reference on m3.xlarge (Exp. C1/C3)
Dockerized Go (Exp. C2 compared with C1) on m3.xlarge
Dockerized Dart (Exp. C4 compared with C3) on m3.xlarge
(3) Container
(1) Container impact might be language dependent.
(2) Container impact is most of the times below 10%.

13. Overlay Network and VM Type
Language impact is more severe than SDN
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
13
Data transfer in relative comparison
(bigger is better)
Message Size (kB)
Ratio(%)
0kB 50kB 150kB 250kB 350kB 450kB
40%60%80%100%120% ●
●
●
●
Reference: (Exp. V1, V3, V5)
SDN loss on m3.large (2−core, V2 compared with V1)
SDN loss on m3.xlarge (4−core, V4 compared with V3)
SDN loss on m3.2xlarge (8−core, V6 compared with V5)
(4) VM Types
Decreasing loss
with increase of
cores due to a
reduction of
CPU contention
effects.
(1) SDN deployed on high core machines decreases performance down to 80% - 90%.
(2) That is far less than a potential language impact (40% - 190%).

14. Conclusion
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
14
• ppbench is a benchmarking tool designed to be
used in the upfront of cloud-native application
design processes.
• ppbench is provided via GitHub:
https://github.com/nkratzke/pingpong
• You can run arbitrary experiments with ppbench
to inspect possible performance impacts in the
upfront of a service design/implementation.
• We presented some evaluation results of
ppbench
• SDN should always be used with high core machine types
• Programming Language might be more performance
relevant than a SDN

15. Acknowledgement
• Stonehenge: Wikipedia, http://pt.wikipedia.org/wiki/Stonehenge
• Soldier: Pixabay (CC0 Public Domain)
• Microservices: Robert Morschel, http://www.soa-probe.com/2015/03/microservices-
summary.html
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
15
This study was funded by German Federal Ministry of Education and Research (Project Cloud TRANSIT,
03FH021PX4). The author thanks Lübeck University (Institute of Telematics) and fat IT solution GmbH (Kiel)
for their support of Cloud TRANSIT.
Picture Reference

16. About
Prof. Dr. rer. nat. Nane Kratzke
Computer Science and Business Information Systems
16
Nane Kratzke
CoSA: http://cosa.fh-luebeck.de/en/contact/people/n-kratzke
Blog: http://www.nkode.io
Twitter: @NaneKratzke
GooglePlus: +NaneKratzke
LinkedIn: https://de.linkedin.com/in/nanekratzke
GitHub: https://github.com/nkratzke
ResearchGate: https://www.researchgate.net/profile/Nane_Kratzke
SlideShare: http://de.slideshare.net/i21aneka