A quick comparison between Tomcat, Undertow, Jetty, Nginx Unit regarding features, performance, scalability, security, maintainability and extensibility

1. Tomcat, Undertow, Jetty,
Nginx Unit
Pros and cons
1

2. • Features
• Performance
• Maintainability
• Security
• Extensibility
• Summary
2
Agenda

3. • The following Java specifications were considered for this analysis:
• Servlet (JSR 369)
• JSP (JSR 152)
• Expression Language (JSR 341)
• Web Sockets (JSR 356)
• Java NIO/NIO.2 (JSR 203)
• HTTP 2 (JEP 110)
3
Features

4. Features
4
Click to edit
Tomcat Undertow Jetty Nginx Unit
Servlet 4.0 4.0 4.0 3.1
JSP 2.3 2.3 2.3 2.3
Expression Language 3.0 3.0 3.0 3.0
Web Sockets 1.1 1.1 1.1 1.1
Java NIO/NIO.2 ✅ ✅ ✅ C backend
HTTP 2 ✅ ✅ ✅ ❌

5. • What is Performance?
• What is Scalability?
• Horizontal/Vertical Scalability
• Summary
5
Performance

6. • There are several definitions for performance
In here, the throughput definition used
6
What is Performance?
“Premature optimization is the root of all evil.”
- Donald Knuth
“Throughput is the rate of
production or the rate at which
something is processed.”
- Wikipedia on Throughput

7. • Any system is a network of queues
7
What is Performance?

8. 8
What is Scalability?
“Scalability is the property of a
system to handle a growing
amount of work by adding
resources to the system.”
- Wikipedia on Scalability

9. 9
Horizontal/Vertical Scalability
“Horizontal scaling means that
you scale by adding more machines
into your pool of resources whereas
Vertical scaling means that
you scale by adding more power
(CPU, RAM) to an existing machine.”
- Stack Overflow answer on Scalability

10. • In general, they have the same performance
• Nginx Unit is still in beta stage as in November 2019
• But it is very promising (multiple programming languages, DPDK, …)
• Jetty, Undertow and Tomcat have some minor performance
differences, but configuration can mitigate it
10
Performance Summary
Tomcat Undertow Jetty Nginx Unit
Keep Tomcat if already using it
Always configure NIO.2
Use Undertow/WildFly (JBoss) if
your webapp requires full JEE
support
Use Jetty when hardware
resources are limited
Not enough data
to recommend

11. • What is software maintainability?
• Used metrics
• Documentation
• Lines of code
• Test coverage
• Summary
11
Maintainability

12. “The ease with which a software
system or component can be
modified to correct faults,
improve performance or other
attributes, or adapt to a changed
environment.”
- IEEE Glossary of Software Eng.
12
What is Software Maintainability?
“The maintainability of software
depends on a few different factors.
In general, it must be easy to
understand the software
(how it works, what it does,
and why it does it the way it does),
easy to find what needs to be
change, easy to make changes
and easy to check
that the changes have not
introduced any bugs.”
- UK Software Sustainability Institute

13. • All projects are well documented
• Tomcat is the winner here once it is the most popular web
application server with many years of documented troubleshooting
including Undertow/Wildfly
13
Documentation

14. • Lines of code is a classic maintainability metric
14
Lines of Code
Total Lines of Code
Tomcat
(9.0.27)
Undertow
(Wildfly 18.0.0)
Jetty
(9.4.22)
Nginx Unit
(master branch 20191023)
430.507
820.318
(contains JEE apis too)
565.122 94.925
“All code is technical debt;
Some code just has a higher interest rate.”
- Paul McMahon

15. • Tomcat and Nginx Unit does not follow the maven project structure
• Undertow and Jetty are maven projects with a cleaner structure and
code base
• Test Coverage could not be checked because each web application
server has a different approach on test
• Tomcat has many tests against regressions
• Nginx Unit has many tests against java specifications (the JSR-XXX)
• Undertow and Jetty use standard maven for most of the components that
are covered but not all (as in 2019-10-23)
15
Test Coverage

16. 16
Maintainability Summary
• Tomcat has more documentation on troubleshooting
• Undertow, Jetty and Nginx Unit have a much cleaner code case with
more coverage

17. • OWASP has several general and specific recommendations
• Common Vulnerabilities and Exposures (CVE) are considered here
17
Security

18. Tomcat Vulnerabilities
18
Click to editYear
# of
Vulnerabiliti
es
DoS
Code
Execution
Overflow XSS
Directory
Traversal
Http
Response
Splitting
Bypass
something
Gain
Information
Gain
Privileges
CSRF
2016 12 2 1 2 5 1 3 1
2017 22 1 3 1 1 1 5 7 2
2018 8 1 1
2019 4 1 1
% Of All 16.4 4.1 3.5 13.5 7.0 0.0 20.5 18.7 4.1 2.3
Source: https://www.cvedetails.com/product/887/Apache-Tomcat.html

19. WildFly Vulnerabilities
19
Click to editYear
# of
Vulnerabiliti
es
DoS
Code
Execution
Overflow XSS
Directory
Traversal
Http
Response
Splitting
Bypass
something
Gain
Information
Gain
Privileges
CSRF
2016 2 1 1
2017 1 1
2018 2 1 1
2019
(N/A)
% Of All 22.2 0.0 11.1 0.0 11.1 11.1 0.0 22.2 0.0 11.1
Source: https://www.cvedetails.com/product/27107/Redhat-Jboss-Wildfly-Application-Server.html

20. Jetty Vulnerabilities
20
Click to editYear
# of
Vulnerabiliti
es
DoS
Code
Execution
Overflow XSS
Directory
Traversal
Http
Response
Splitting
Bypass
something
Gain
Information
Gain
Privileges
CSRF
2016 1 1
2017 2 1 1
2018 5 1 2 1
2019 4 1 1 2
% Of All 8.3 0.0 8.3 8.3 0.0 0.0 25.0 41.7 0.0 0.0
Source: https://www.cvedetails.com/product/34824/Eclipse-Jetty.html

21. 21
Security Summary
• There are no CVE data regarding Nginx Unit (as in November 2019)
• Once it’s written in C, we can infer it may have a greater number of
vulnerabilities than a pure java-based web application server
• Tomcat is clearly more vulnerable than the others
• Jetty comes next
• Undertow/Wildfly is the least vulnerable but there is not data for
2019

22. • Servlet Filter
• Application/Container wide Filter
• Tomcat Valve
• Undertow HttpHandler
• Jetty Handler
• Nginx Unit Routes
22
Extensibility

23. 23
Servlet Filter
What is a filter?
“A filter is a reusable piece of code that can
transform the content of HTTP requests,
responses, and header information.
Filters do not generally create a response or
respond to a request as servlets do, rather they
modify or adapt the requests for a resource,
modify or adapt responses from a resource.”
- Java Servlet 4.0 specification

24. 24
Tomcat Valve
What is a Tomcat Valve?
“A Valve element represents
a component that will be inserted
into the request processing pipeline
for the associated Catalina container
(Engine, Host, or Context).
Individual Valves have distinct
processing capabilities (…)”
- Tomcat documentation

25. 25
Undertow HttpHandler
Undertow HttpHandler Behaviour
“Handlers are generally chained together by
explicitly specifying the next handler at
construction time, there is no pipeline concept,
which means that a handler can pick the next
handler to invoke based on the current request.”
- Undertow documentation

26. 26
Jetty Handler
Jetty Handler Behaviour
“The Handler is the Jetty
component that deals with
HTTP requests and responses.”
- Jetty documentation

27. Nginx Unit Routes Behaviour
“Nginx Unit configuration offers a routes
object to enable elaborate internal
routing between listeners and apps.
Listeners pass requests to routes
or directly to apps.
Requests are matched against route
step conditions; a request matching all
conditions of a step is passed to the app
or the route that the step specifies.”
- Nginx Unit documentation
27
Nginx Unit Routes

28. 28
Extensibility Summary
• Nginx Unit is a clear winner once the route configuration forces the
web application decoupling
• Tomcat, Undertow, Jetty are equivalent on extensibility but their
architecture are very different

29. 29
Summary
Tomcat Undertow Jetty Nginx Unit
Features
Tomcat, Undertow and Jetty are equivalent
Use Undertow if JEE support is required
Still in development
Performance
Keep Tomcat if you
already have it
Undertow has an
equivalent performance
with Tomcat
Use Jetty for resource
constrained
environments such as
cloud
Not enough data to
recommend
Maintainability
Tomcat wins on
documentation and
troubleshooting
Undertow, Jetty and Unit are pretty much equivalent on maintainability with a
much cleaner code base
Security
Tomcat has more
vulnerabilities than the
others
Undertow is the least
vulnerable but there is no
data for 2019
Jetty is the second most
vulnerable
No CVEs for Unit
Extensibility
Tomcat, Undertow and Jetty are equivalent
Unit has a better approach by proposing decouple web applications to talk to each other with routes

30. • JSR 152: JavaServer Pages Specification
• JSR 203: Java NIO/NIO.2
• JSR 341: Expression Language
• JSR 356: Java API for WebSocket
• JSR 369: Java Servlet Specification
• JSR 372: JavaServer Faces Specification
• JEP 110: HTTP/2 Client
30
References

31. • Apache Tomcat
• Undertow
• Jetty
• Nginx Unit
• Tomcat vs. Jetty vs. Undertow: Comparison of Spring Boot Embedded
Servlet Containers
• Top Java Application Servers: Tomcat vs. Jetty vs. GlassFish vs. WildFly
• JBoss vs. Tomcat: Choosing A Java Application Server
• Tomcat vs Jetty vs Wildfly runtime comparision
31
References

32. • Low level programming
Igor Zhirkov
Apress
• Systems Performance – enterprise and the cloud
Brendan Gregg
Prentice Hall
• Performance modeling and design of computer systems
Mor Harchol-Balter
Cambridge University Press
• Operating System Concepts
Silberschatz, Gagne, Galvin
John Wiley & Sons
32
References

33. 33
Thank you!
Geraldo Netto
geraldo.netto@gmail.com