The document discusses using OpenSSL to improve performance in Tomcat. It describes Tomcat connectors like NIO, NIO2, and APR and the new OpenSSL implementation. Performance tests show the OpenSSL implementation outperforms JSSE and has throughput similar to NIO and NIO2. APR is not needed. OpenSSL is currently needed for HTTP/2 support until Java 9. The presentation concludes that OpenSSL significantly boosts performance over JSSE.

1. Using OpenSSL to boostUsing OpenSSL to boost
TomcatTomcat
Jean-Frederic ClereJean-Frederic Clere

2. What I will coverWhat I will cover
●
Who I am.
●
Connectors
– NIO, NIO2, APR
– OpenSSLImplementation
– HTTP/2 and ALPN in Tomcat.
●
Performance tests
– With ab and h2load as client load generator.
●
Questions?
12/16/16 2

3. Who I amWho I am
Jean-Frederic Clere
Red Hat
Years writing JAVA code and server software
Tomcat committer since 2001
Doing OpenSource since 1999
Cyclist/Runner etc
Lived 15 years in Spain (Barcelona)
Now in Neuchâtel (CH)
12/16/16 3

4. TomcatTomcat
12/16/16 4

5. What is a Connector?What is a Connector?
● Tomcat's interface to the world
● Binds to a port
● Understands a protocol and possible upgrades.
● Dispatches requests (example)
– protocol="org.apache.coyote.http11.Http11AprProtocol"
– protocol="org.apache.coyote.http11.Http11NioProtocol"
– protocol="org.apache.coyote.http11.Http11Nio2Protocol"
12/16/16 5

6. Tomcat ConnectorsTomcat Connectors
● Java Non-blocking I/O (NIO)
● Native / Apache Portable Runtime (APR)
● Java NIO.2
Technically, there are combinations of all of the above with HTTP and AJP protocols.
The presentation focuses on HTTP and on NIO/NIO2.
12/16/16 6

7. What is new in Tomcat 9 / 8.5What is new in Tomcat 9 / 8.5
● Property sslImplementationName
– Allows replacement of the SSL code
● OpenSSLImplementation (use OpenSSL)
● JSSEImplementation (use JSSE)
● UpgradeProtocol
– Allows protocol upgrade from HTTP/1.1
● HTTP/2 (yes)
● Websocket (cool) / Speedy (no plan to support it).
12/16/16 7

8. Why a new SSLImplementationWhy a new SSLImplementation
● JSSE:
– Very slow
– Missing features: like ALPN (JEP 244: TLS Application-Layer Protocol Negotiation)
– Hardware acceleration very partial (like AES in java8)
● Native connector:
– Fast but a lot of native code
– Use OpenSSL for SSL/TLS.
● New OpenSSL implementation:
– Fast.
– Uses only a OpenSSL for native code (no native socket, poller etc).
– Works with NIO and NIO2.
– Uses OpenSSL for SSL/TLS. (warp, unwarp, handshake etc).
12/16/16 8

9. OpenSSLImplementationOpenSSLImplementation
● Code originates from netty-tcnative a forked Tomcat Native
● Prototype (last year):
– Done with the BeFriNe University
– Tested and ported to tc_trunk last summer
● SSL Configuration compatible with the JSSE connection (*)
● Uses keystores (*)
● Uses SSL BIO to wrap/unwarp, handshake
● Uses java NIO or NIO2 Sockets for the reads and writes
● Automatically enabled when TC native is installed/enabled (*)
12/16/16 9

10. How TLS is done in TomcatHow TLS is done in Tomcat
12/16/16 10
Tomcat
JSSE Con.
Javastdlib
JSSE SSL Engine
NIO/NIO2
Tomcat Native
APR JNIs
Webserver
APR Internals
APR Connector
OpenSSL OS Sockets
JavaC/Native
Webserver
OpenSSL Impl.

11. How does that worksHow does that works
SSLContext
JSSESSLContext OpenSSLContext
SSLEngine
SSLContext
OpenSSLEngine
createSSLEngine() createSSLEngine()
wrap()
unwrap()
getSession()
etc...
Overrides
12/16/16 11

12. How does wrap worksHow does wrap works
wrap(plaintext, encrypted)
internalBIO networkBIO
BIO_new_bio_pair
SSL_set_bio
writePlainTextData
write_ToSSL
SSL_write
readEncryptedData
readFromBIO
BIO_read
12/16/16 12

13. How does unwrap worksHow does unwrap works
unwrap(encrypted, plaintext)
internalBIO networkBIO
BIO_new_bio_pair
SSL_set_bio
writeEncryptedData
writeToBIO
BIO_write
readPlaintextData
readFromSSL
SSL_read
12/16/16 13

14. Connector PerformanceConnector Performance
● Compare connectors throughput against each other
● Only static content was compared, varying file sizes
● Run on “fast” machines, 10 Gbps local network
● Tests:
– Compare the connectors (trunk) NIO, NIO2 and APR
– Using JSSE and OpenSSL
– First without “sendfile”
12/16/16 14

15. Connector Throughput (c8)Connector Throughput (c8)
4KiB.bin
8KiB.bin
16KiB.bin
32KiB.bin
64KiB.bin
128KiB.bin
256KiB.bin
512KiB.bin
1MiB.bin
2MiB.bin
4MiB.bin
8MiB.bin
16MiB.bin
32MiB.bin
0
100000
200000
300000
400000
500000
600000
700000
Concurency 8
coyote_apr_https
coyote_nio2_openssl_https
coyote_nio_jsse_https
coyote_nio_openssl_https
File Size
ThroughputKbytes/sec
12/16/16 15

16. Connector Throughput (c40)Connector Throughput (c40)
4KiB.bin
8KiB.bin
16KiB.bin
32KiB.bin
64KiB.bin
128KiB.bin
256KiB.bin
512KiB.bin
1MiB.bin
2MiB.bin
4MiB.bin
8MiB.bin
16MiB.bin
32MiB.bin
0
100000
200000
300000
400000
500000
600000
700000
Concurency 40
coyote_apr_https
coyote_nio2_openssl_https
coyote_nio_jsse_https
coyote_nio_openssl_https
File Size
ThroughputKbytes/sec
12/16/16 16

17. Connector Throughput (c80)Connector Throughput (c80)
4KiB.bin
8KiB.bin
16KiB.bin
32KiB.bin
64KiB.bin
128KiB.bin
256KiB.bin
512KiB.bin
1MiB.bin
2MiB.bin
4MiB.bin
8MiB.bin
16MiB.bin
32MiB.bin
0
100000
200000
300000
400000
500000
600000
700000
concurency 80
coyote_apr_https
coyote_nio2_openssl_https
coyote_nio_jsse_https
coyote_nio_openssl_https
File Size
ThroughputKbytes/sec
12/16/16 17

18. Connector CPU UseConnector CPU Use
4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB
40
50
60
70
80
90
100
Concurency 8
4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB
40
50
60
70
80
90
100
concurency 40
4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB
40
50
60
70
80
90
100
Concurency 80
coyote_apr_https
coyote_nio2_openssl_https
coyote_nio_jsse_https
coyote_nio_openssl_https
File Size
CPUusage
12/16/16 18

19. Connector TC8.5Connector TC8.5
12/16/16 19
4KiB.bin
8KiB.bin
16KiB.bin
32KiB.bin
64KiB.bin
128KiB.bin
256KiB.bin
512KiB.bin
1MiB.bin
0
100000
200000
300000
400000
500000
600000
700000
800000
Concurency 320 tomcat 8.5
coyote_apr_https
coyote_nio_jssehttps
coyote_nio_opensslhttps
File Size
Kbytes/second
4KiB
8KiB
16KiB
32KiB
64KiB
128KiB
256KiB
512KiB
1MiB
0
20
40
60
80
100
120
Concurency 320 tomcat8.5
coyote_apr_https
coyote_nio_jssehttps
coyote_nio_opensslhttps
File Size
CPUusage

20. Connector PerformanceConnector Performance
● With sendfile
– In fact with TLS/SSL sendfile is emulated
12/16/16 20

21. Connector Throughput (c8)Connector Throughput (c8)
4KiB.bin
8KiB.bin
16KiB.bin
32KiB.bin
64KiB.bin
128KiB.bin
256KiB.bin
512KiB.bin
1MiB.bin
2MiB.bin
4MiB.bin
8MiB.bin
16MiB.bin
32MiB.bin
0
100000
200000
300000
400000
500000
600000
700000
800000
Concurency 8
coyote_apr_https
coyote_nio2_openssl_https
coyote_nio_jsse_https
coyote_nio_openssl_https
File Size
ThroughputinKbytes/sec
12/16/16 21

22. Connector Throughput (c40)Connector Throughput (c40)
4KiB.bin
8KiB.bin
16KiB.bin
32KiB.bin
64KiB.bin
128KiB.bin
256KiB.bin
512KiB.bin
1MiB.bin
2MiB.bin
4MiB.bin
8MiB.bin
16MiB.bin
32MiB.bin
0
100000
200000
300000
400000
500000
600000
700000
800000
Concurency 40
coyote_apr_https
coyote_nio2_openssl_https
coyote_nio_jsse_https
coyote_nio_openssl_https
File Size
ThroughputinKbytes/sec
12/16/16 22

23. Connector Throughput (c80)Connector Throughput (c80)
4KiB.bin
8KiB.bin
16KiB.bin
32KiB.bin
64KiB.bin
128KiB.bin
256KiB.bin
512KiB.bin
1MiB.bin
2MiB.bin
4MiB.bin
8MiB.bin
16MiB.bin
32MiB.bin
0
100000
200000
300000
400000
500000
600000
700000
800000
Concurency 80
coyote_apr_https
coyote_nio2_openssl_https
coyote_nio_jsse_https
coyote_nio_openssl_https
File Size
ThroughtinKbytes/sec
12/16/16 23

24. Connector CPU UseConnector CPU Use
4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB
40
50
60
70
80
90
100
Concunreny 8
4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB
40
50
60
70
80
90
100
Concurency 40
4KiB 16KiB 64KiB 128KiB 512KiB 2MiB 8MiB 32MiB
40
50
60
70
80
90
100
Concurency 80
coyote_apr_https
coyote_nio2_openssl_https
coyote_nio_jsse_https
coyote_nio_openssl_https
12/16/16 24

25. Connector PerformanceConnector Performance
● Conclusion:
– OpenSSL performs better that JSSE
– NIO and NIO(2) give similar results
– Emulated sendfile doesn't help a lot (bigger files better).
– APR isn't needed
– Until Java9 is released OpenSSL is needed for HTTP/2
12/16/16 25

26. Questions?Questions?
Thank you!Thank you!
● jfclere@gmail.com
● users@tomcat.apache.org
● Repo with the scripts for the tests:
– https://github.com/jfclere/AC2014scripts
12/16/16 26

27. Jean-Frederic Clere
@jfclere
jfclere@gmail.com