- Designed and developed simplified version of single thread event-based web server engine with Java - Designed overall architectures and implemented event loop and socket programming - Attained higher amount of throughput and lower error rate of http request handling comparing with NodeJS

1. Event loop 를 이용한 웹 엔진 개발
17.6.16. 최종발표
F조 이기웅, 유종민, 김수

2. 2
Results & Demo
1. Background
2. Goal
3. Diagram
3. Issue
4. Demo

3. 3
Background
Two Types of Web Server
Multi-Threaded Server
Single-Threaded Server

4. Background
Multi-Threaded Server Shortcomings
• Spawn a new thread for every new http requests
• Overhead of creating a new thread for every requests with simultaneous users access
• Memory allocation with threads
• Overhead with context switching
• In I/O intensive scenarios the requests spend most of the time waiting for I/O to complete
Multi-Threaded Server Shortcomings

5. Background
Single Thread Server : Alternative of Multi-Thread Server
Single-Threaded Server
• Asynchronous event driven IO which uses thread pool for IO task handling
• Events made from http requests is dealt by Handler/Listener with callback
• Application not to pause or sleep, but to become available for other requests.
• Suitable for implementing Web Server which has Intensive IO task with clients

6. 6
Goal
Goal
• Implementing Web Server Engine which deals with several http requests
Requirements with Stable/Fast Web Server Engine
• Stability - Handle large amounts of simultaneous http requests without crashes
• Performance - High Latency, Low Error Rates
• Reliability - Prevent malicious/abnormal client access
Subgoals
• Implement event loop structure and communication capabilities between event loop/worker threads
• Caching/Buffer to improve performance with IO
• Prevent malicious client access with two representative cases
• Performance comparison with existing web server engines

7. 7
Diagram
서버 주요 모듈

8. 8
Diagram
서버 주요 모듈 - Selector + Event Loop
Selector - Select
Initialization with opening socket
to communicate with clients and
registering server to selector
waiting for requests from clients
Selector - Accept/Read/Write
Selector periodically checking
requests from clients, register
clients to selector and handle
request as an event in event loop
Ready Key(Selected Key)
Selected key retrieved
from selector, which is an event
transformed from client request

9. 9
Diagram
서버 주요 모듈 - HttpParser + Request Processor
HttpParser - parseRequest
Http content from a client
is parsed into hash table,
returning status of http
request
Request Processor - process
If status is 200, keep handling
the process.Otherwise make a
http response with error code
to the client
Error Code
304 - Not Modified
400 - Bad Request
404 - Not Found
500 - Interval Server Error

10. 10
Diagram
서버 주요 모듈 - Event Loop
Request Processor - process
With http request status 200,
if task is light-weight overload
handle the task in event loop.
Otherwise, handle task with
additional async thread
Main Controller - writer
With light-weight overload task
after handling it in process,
make a http response to client
In a format of http message

11. 11
Diagram
서버 주요 모듈 - I/O Thread + Thread Pool + Mem Cache
I/O Thread - handle
From the thread pool, allocate
file IO task to waiting thread
by calling submit function
I/O Thread - loadFile
Following the filePath, read the
file from memory cache if exists.
Otherwise read file from disk.
MainController - writer
Make a http response to client
in a format of http message

12. 12
Issues
Web Cache Replacement Policy에 따른 Performance Optimization
Problem
Latency inefficiency with file I/O task handling
Solution
Find an optimal replacement policy for web cache,
and apply it to memory cache of web server
1.FIFO(First In First Out)
2. LRU(Least Recently Used)
3. LFU(Least Frequently Used)

13. 13
Issues
Web Cache Optimization - Zipf distribution
Zipf Distribution
- A few elements that score very high
(the left tail in the diagrams)
- A medium number of elements with
middle-of-the-road scores
(The middle part of the diagram)
- A huge number of elements that
score very low
(The right part of the diagram)
* LRU Suitable than LFU

14. 14
Issues
Web Cache optimization - Test with LFU/LRU
• LFU - 1,000 Simultaneous clients requests
• LRU - 1,000 Simultaneous clients requests
There was not much
difference between
LFU and LRU Policy
=> Adopted LRU on
caching eviction
14

15. 15
Issues
ByteBuffer Optimization - Kernel Buffer/Heap Buffer
Problem
Reading http request message from clients and writing http response to client,
It is essential to read/write message in a byte buffer. Consuming lots of times,
it becomes one of the reason for delayed latency.
Solution
While considering pros and cons among kernel buffer and heap buffer, apply one
with better performance to obtain optimization.
1.Kernel Buffer
2. Heap Buffer

16. 16
Issues
ByteBuffer Optimization - Kernel Buffer/Heap Buffer
Kernel Buffer Heap Buffer
Definition
Contents of direct buffers reside
outside of the normal garbage-
collected heap
Within JVM, memory where
objects are stored
Pros
Faster to read/write
on memory for filling
With bytes and integers
Allocation and deallocation is
less expensive. GC enabled
Cons
Allocation and deallocation is
more expensive
Slower than kernel buffer due
to GC functionality

17. 17
Issues
ByteBuffer Optimization - Kernel Buffer/Heap Buffer

18. 18
Issues
ByteBuffer Optimization - Kernel Buffer/Heap Buffer
• Heap Buffer - 1,000 Simultaneous clients requests
• Kernel Buffer - 1,000 Simultaneous clients requests
Kernel Buffer outperforms
Heap buffer in aspects of
error rate and throughput.
=> Adopted kernel buffer
by allocateDirect during
initialization of program

19. 19
Issues
Reliability - Malicious/Abnormal Client Request
Problem
Web server confronts various malicious programs/scripts from hackers, at the same
time abnormal client requests. This leads to server crash, so it needs to be prevented.
Problem 1 - Request timeout.
Problem 2 - Request too long.
Solution
Due to the fact that there are numerous kinds of malicious attacks/abnormal requests,
prevent two representative cases by reflecting proper handling logics into the code.
Solution 1 - Register request timeout limit on http request, return 408 error
Solution 2 - Limit buffer size read from client http request, return 400 error

20. 20
Issues
Reliability - Malicious/Abnormal Client Request
• Request timeout
• Request too long
Find request
timeout problem
and return 408
Request timeout
Find request
too long problem
and return 400
Bad Request

21. 21
Issues
Simultaneous multi-client request optimization - Backlog
Problem
Java 기반의 시스템은 메모리를 JVM(Java Virtual Machine)에서 관리하기 때문에 Full GC
발생으로 인한 STW(Stop The World) 현상에서 자유로울 수 없다. 또한 Full GC 발생 시의
휴지기가 1초에서 4초 이상으로 급격하게 늘어, 시스템이 한 동안 먹통이 되면
모든 요청에 대해 응답할 수 없는 상태에 빠지게 된다.
Solution
Backlog는 미연결 연결들에 대한 큐의 늘어날 수 있는 최대 길이를 정의한다. 전체 시스템
메모리와 Asynchronous Web Server가 차지하는 Heap의 크기 및 Client당 메모리 점유에 대한
계산을 통해 적절한 Backlog와의 설정이 필요하다.
따라서 Backlog양을 50부터 200까지 10씩 늘려가며 profiling한 결과 최적의 결과값을
보여주는 150으로 설정하였다.

22. 22
Issues
Simultaneous multi-client request optimization - Backlog
• Backlog 50
• Backlog 150
Backlog of 150 outperforms
backlog of 50 in aspects of
error rate and throughput.
=> Adopted backlog 150
in server socket binding

23. 23
Live Demo - Open Browser and apply web server engine onto it
Live Demo

24. 24
Performance Measure
With JMeter
• NodeJS 및 타 비동기 웹 서버와의 비교 기준
• Test Plan #1 : 동시접속자 100명이 1MB 이하
의 File I/O를 총 1000번 정도의 Request
• Criteria : Average Throughput, Average
Response Time, Error Rate
• Test Plan #2 : 동시접속자 1000명이 1MB 이
하의 File I/O를 총 10000번 정도의 Request
• Criteria : Test Plan #1와 동일

25. 25
Live Demo - JMeter Test
Live Demo

26. 26
Division and Assignment of Work
• 김수
- 개발 환경 구축 및 Github 초기 연동 작업
- Http Parser 및 File IO Thread 구현
- Http 프로그래밍 관련 주요 이슈 담당
- 전체 문서 관리
- Caching(LFU/LRU/FIFO) 구현 및 테스트 담당
- Event Loop와 Thread Pool 통신 기능 구현
- JMeter에 의한 테스트셋 구현 및 성능 향상 담당
- 일정 조정 및 커뮤니케이션 담당
- Event Loop 구현 담당
- 소켓 프로그래밍 관련 주요 이슈 담당
• 유종민
• 이기웅

27. 27
Schedule
프로젝트 시작
교수님 면담
3/3
담당자 미팅
3차 구현 완료
스펙 발표
3/31
최종 발표
6/16 2차 구현 완료
1차 구현 완료
6/16
담당자 미팅
3/10
6/9
교수님 면담
3/17
셀렉터 공부
3/16
최적화 진행
중간 발표
5/19
로직 설계
4/3

28. End of Document
감사합니다.
F 조 김수, 유종민, 이기웅 010.9553.2500. wooong1225@gmail.com