SlideShare a Scribd company logo
Tips on High Performance
   Server Programming
        Joshua Zhu
       June 9, 2009
Agenda
•   Fundamentals
•   Best practices and common tricks
•   Design issues
•   Tools and resources
Key Rules
• Do NOT block
• Avoid excessive system calls
• Cache/preprocess as much as possible
• Use efficient algorithms and data structures
• Threads are usually a bad idea
• Separate the I/O code from the business-logic
  code
• Keep the most heavily used data near the CPU
• Tune against the bottleneck
I/O Models
•   Blocking
•   Non-blocking
•   I/O multiplexing
•   Signal-driven I/O
•   Asynchronous I/O
I/O Multiplexing
•   Select
•   Poll
•   /dev/poll
•   Epoll/kqueue
    – Level triggered
    – Edge triggered
I/O Strategies
•   1 thread, non-blocking, level-triggered
•   1 thread, non-blocking, edge-triggered
•   1 thread, AIO
•   1 thread per client
•   Build the server code into the kernel
Let’s Face the C10K Problem
• 10000 connections
  – CPU/memory usage per connection
• 10000 hits/sec
  – 10 us per hit
     • Instruction (ns)
     • System call (us)
Agenda
•   Fundamentals
•   Best practices and common tricks
•   Design issues
•   Tools and resources
The Event-driven Model
while (true) {
     for t in run_tasks:
               t.handler();

     update_time(&now);
     timeout = ETERNITY;

     for t in wait_tasks: /* sorted already */
              if (t.time <= now) {
                             t.timeout_handler();
              } else {
                             timeout = t.time - now;
                             break;
              }

     nevents = poll_function(events, timeout);
     for i in nevents:
               task t;

              if (events[i].type == READ) {
                             t.handler = read_handler;
              } else (events[i].type == WRITE) {
                             t.handler = write_handler;
              }

              run_tasks_add(t);
}
Scheduler
• Task scheduling
  – Run queue
  – Wait queue
• Timers
  – Time jumps
  – The next timeout
     • Pass to select()/poll()/epoll_wait()...
  – Data structures
     • Red-black tree
     • Min-heap
     • Timer wheel
Unify Multiple Event Sources
• I/O events
• Timer events
• Signals/threads
  – Use a pipe or socketpair
    • Register the read end in the event loop
    • Notify the event loop by writing to the write end
Buffer Management
• Fixed size or not
• R/W pointers operation
  – Producer/consumer
• Circular buffer
• Single buffering
• Buffer chain
Memory
• Memory fragment
   – Memory pool
      • Fixed size
          – MRU
      • Non-fixed size
   – Slab allocator
• Memory operations are expensive
   – Allocation/free
   – Data copies
• Pre-allocation/static-allocation
   – Array
Strings
• Gotchas
  – Strlen()
     • Sizeof() - 1
  – Strncpy()
     • Strlcpy()
  – Vsnprintf()
  –…
• Algorithms
  – KMP/BM
  – AC/WM
  –…
Caching
• Time
  – Reduce the number of gettimeofday() calls
  – Time resolution
     • ms or sec?
• Content
  – Files
     • In-memory buffering
  – Database
     • Memcached
• Preprocessing
Accept() Strategies
• Accept-limit
  – Multi-accept
  – Accept-mutex
• The thundering herd problem
  – Polling functions
  – Kernels
  – The amount of processes/threads
Concurrency
• Threads
  – Pros
     • Utilize all the CPUs
     • True CPU concurrency
  – Cons
     • Context switches
     • Locks hurt performance
           – Deadlocks
           – Starvation
           – Race conditions
     • Error prone and hard to debug
     • Hard limits of operating systems
Concurrency (cont’d)
• Categorize your service
  – CPU-bound
  – I/O-bound
• Models
  – Thread pool
  – Process pool
  – SEDA
  – Master/workers
• CPU affinity
Advanced I/O Functions
• Gather read, scatter write
  – Readv/writev
• Sendfile
• Mmap
• Splice and tee
Take Advantage of TCP/IP Options
• TCP/IP options
  – SO_REUSEADDR
  – SO_RCVBUF/SO_SNDBUF
  – TCP_CORK
  – TCP_NODELAY
  – TCP_DEFER_ACCEPT
  –…
Misc.
• Byte order
  – Little-endian
  – Big-endian
• Max open file number
  – Ulimit/setrlimit
Agenda
•   Fundamentals
•   Best practices and common tricks
•   Design issues
•   Tools and resources
Protocol
• Readability
  – Binary
  – Textual
• Compatibility
• Security
  – Heart beat
  – Invalid data
• The state machine (FSM)
  – Granularity
  – Traceability
Security
•   Timeouts
•   Bad data
•   Buffer overflow
•   DOS attack
•   Encryption
•   Privilege
•   Working directory
Flexibility
• Modularize
  – Modules
  – Plugins
• Scriptable
  – Lua
• Hot code update
• HA
Introspection
• Statistics
  – SNMP
• Debugable
• Tunable
• Logs
Patterns
•   Reactor
•   Proactor
•   Half-Sync/Half-Async
•   Leader/Followers
•   …
Choose a Suitable Language
•   C
•   C++
•   Java
•   Erlang
•   …
The Devil Hides in the Details
• Find out the shortest plank in the bucket
   – Profile/benchmark
• Take care of every line of your code!
   – Return values
   – Error code
      • EINTR
      • SIGPIPE
      • …
• Tune your operating system if necessary
   – /proc
• Make sure hardware is not the bottleneck
   – Network cards
   – Disk
   – …
Agenda
•   Fundamentals
•   Best practices and common tricks
•   Design issues
•   Tools and resources
Sharpen Your Tools
•   Ping
•   Traceroute
•   Netstat
•   Lsof
•   Strace
•   Nc
•   Wireshark
•   Gprof
•   Valgrind
•   Vmstat/iostat/dstat
•   Dot/gnuplot
Understand TCP/IP Well
• Three-way handshake
• Connection tear-down
• TCP state transition
  – TIME_WAIT
  – CLOSE_WAIT
  –…
Learn By Reading Code
• Frameworks
  –   ACE
  –   Libevent/libev
  –   Boost::asio
  –   …
• Servers
  –   HAProxy
  –   Nginx
  –   Lighttpd
  –   Memcached
  –   MySQL Proxy
  –   …
Recommended Books
• Advanced Programming in the UNIX
  Environment
• UNIX Network Programming
• TCP/IP Illustrated
• Effective TCP/IP Programming
• Pattern-Oriented Software Architecture
• C++ Network Programming
• Programming With POSIX Threads
• Introduction to Algorithms
Thank You!
Visit www.zhuzhaoyuan.com for more info

More Related Content

What's hot

Outlook and Exchange for the bad guys
Outlook and Exchange for the bad guysOutlook and Exchange for the bad guys
Outlook and Exchange for the bad guys
Nick Landers
 
Kafka Summit NYC 2017 - Singe Message Transforms are not the Transformations ...
Kafka Summit NYC 2017 - Singe Message Transforms are not the Transformations ...Kafka Summit NYC 2017 - Singe Message Transforms are not the Transformations ...
Kafka Summit NYC 2017 - Singe Message Transforms are not the Transformations ...
confluent
 
[오픈소스컨설팅] 스카우터 사용자 가이드 2020
[오픈소스컨설팅] 스카우터 사용자 가이드 2020[오픈소스컨설팅] 스카우터 사용자 가이드 2020
[오픈소스컨설팅] 스카우터 사용자 가이드 2020
Ji-Woong Choi
 
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
SANG WON PARK
 
Advanced SQL injection to operating system full control (slides)
Advanced SQL injection to operating system full control (slides)Advanced SQL injection to operating system full control (slides)
Advanced SQL injection to operating system full control (slides)
Bernardo Damele A. G.
 
Delivering: from Kafka to WebSockets | Adam Warski, SoftwareMill
Delivering: from Kafka to WebSockets | Adam Warski, SoftwareMillDelivering: from Kafka to WebSockets | Adam Warski, SoftwareMill
Delivering: from Kafka to WebSockets | Adam Warski, SoftwareMill
HostedbyConfluent
 
Bare Metal Cluster with Kubernetes, Istio and Metallb | Nguyen Phuong An, Ngu...
Bare Metal Cluster with Kubernetes, Istio and Metallb | Nguyen Phuong An, Ngu...Bare Metal Cluster with Kubernetes, Istio and Metallb | Nguyen Phuong An, Ngu...
Bare Metal Cluster with Kubernetes, Istio and Metallb | Nguyen Phuong An, Ngu...
Vietnam Open Infrastructure User Group
 
Producer Performance Tuning for Apache Kafka
Producer Performance Tuning for Apache KafkaProducer Performance Tuning for Apache Kafka
Producer Performance Tuning for Apache Kafka
Jiangjie Qin
 
Nginx Essential
Nginx EssentialNginx Essential
Nginx Essential
Gong Haibing
 
톰캣 운영 노하우
톰캣 운영 노하우톰캣 운영 노하우
톰캣 운영 노하우
jieunsys
 
NGINX: Basics and Best Practices
NGINX: Basics and Best PracticesNGINX: Basics and Best Practices
NGINX: Basics and Best Practices
NGINX, Inc.
 
HAProxy
HAProxy HAProxy
HAProxy
Arindam Nayak
 
Postgresql database administration volume 1
Postgresql database administration volume 1Postgresql database administration volume 1
Postgresql database administration volume 1
Federico Campoli
 
Apache BookKeeper State Store: A Durable Key-Value Store - Pulsar Summit NA 2021
Apache BookKeeper State Store: A Durable Key-Value Store - Pulsar Summit NA 2021Apache BookKeeper State Store: A Durable Key-Value Store - Pulsar Summit NA 2021
Apache BookKeeper State Store: A Durable Key-Value Store - Pulsar Summit NA 2021
StreamNative
 
Offzone | Another waf bypass
Offzone | Another waf bypassOffzone | Another waf bypass
Offzone | Another waf bypass
Дмитрий Бумов
 
카프카, 산전수전 노하우
카프카, 산전수전 노하우카프카, 산전수전 노하우
카프카, 산전수전 노하우
if kakao
 
[pgday.Seoul 2022] PostgreSQL with Google Cloud
[pgday.Seoul 2022] PostgreSQL with Google Cloud[pgday.Seoul 2022] PostgreSQL with Google Cloud
[pgday.Seoul 2022] PostgreSQL with Google Cloud
PgDay.Seoul
 
Ceph issue 해결 사례
Ceph issue 해결 사례Ceph issue 해결 사례
Ceph issue 해결 사례
Open Source Consulting
 
Wireless Cracking using Kali
Wireless Cracking using KaliWireless Cracking using Kali
Wireless Cracking using Kali
n|u - The Open Security Community
 
Secrets of Performance Tuning Java on Kubernetes
Secrets of Performance Tuning Java on KubernetesSecrets of Performance Tuning Java on Kubernetes
Secrets of Performance Tuning Java on Kubernetes
Bruno Borges
 

What's hot (20)

Outlook and Exchange for the bad guys
Outlook and Exchange for the bad guysOutlook and Exchange for the bad guys
Outlook and Exchange for the bad guys
 
Kafka Summit NYC 2017 - Singe Message Transforms are not the Transformations ...
Kafka Summit NYC 2017 - Singe Message Transforms are not the Transformations ...Kafka Summit NYC 2017 - Singe Message Transforms are not the Transformations ...
Kafka Summit NYC 2017 - Singe Message Transforms are not the Transformations ...
 
[오픈소스컨설팅] 스카우터 사용자 가이드 2020
[오픈소스컨설팅] 스카우터 사용자 가이드 2020[오픈소스컨설팅] 스카우터 사용자 가이드 2020
[오픈소스컨설팅] 스카우터 사용자 가이드 2020
 
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
Apache kafka 모니터링을 위한 Metrics 이해 및 최적화 방안
 
Advanced SQL injection to operating system full control (slides)
Advanced SQL injection to operating system full control (slides)Advanced SQL injection to operating system full control (slides)
Advanced SQL injection to operating system full control (slides)
 
Delivering: from Kafka to WebSockets | Adam Warski, SoftwareMill
Delivering: from Kafka to WebSockets | Adam Warski, SoftwareMillDelivering: from Kafka to WebSockets | Adam Warski, SoftwareMill
Delivering: from Kafka to WebSockets | Adam Warski, SoftwareMill
 
Bare Metal Cluster with Kubernetes, Istio and Metallb | Nguyen Phuong An, Ngu...
Bare Metal Cluster with Kubernetes, Istio and Metallb | Nguyen Phuong An, Ngu...Bare Metal Cluster with Kubernetes, Istio and Metallb | Nguyen Phuong An, Ngu...
Bare Metal Cluster with Kubernetes, Istio and Metallb | Nguyen Phuong An, Ngu...
 
Producer Performance Tuning for Apache Kafka
Producer Performance Tuning for Apache KafkaProducer Performance Tuning for Apache Kafka
Producer Performance Tuning for Apache Kafka
 
Nginx Essential
Nginx EssentialNginx Essential
Nginx Essential
 
톰캣 운영 노하우
톰캣 운영 노하우톰캣 운영 노하우
톰캣 운영 노하우
 
NGINX: Basics and Best Practices
NGINX: Basics and Best PracticesNGINX: Basics and Best Practices
NGINX: Basics and Best Practices
 
HAProxy
HAProxy HAProxy
HAProxy
 
Postgresql database administration volume 1
Postgresql database administration volume 1Postgresql database administration volume 1
Postgresql database administration volume 1
 
Apache BookKeeper State Store: A Durable Key-Value Store - Pulsar Summit NA 2021
Apache BookKeeper State Store: A Durable Key-Value Store - Pulsar Summit NA 2021Apache BookKeeper State Store: A Durable Key-Value Store - Pulsar Summit NA 2021
Apache BookKeeper State Store: A Durable Key-Value Store - Pulsar Summit NA 2021
 
Offzone | Another waf bypass
Offzone | Another waf bypassOffzone | Another waf bypass
Offzone | Another waf bypass
 
카프카, 산전수전 노하우
카프카, 산전수전 노하우카프카, 산전수전 노하우
카프카, 산전수전 노하우
 
[pgday.Seoul 2022] PostgreSQL with Google Cloud
[pgday.Seoul 2022] PostgreSQL with Google Cloud[pgday.Seoul 2022] PostgreSQL with Google Cloud
[pgday.Seoul 2022] PostgreSQL with Google Cloud
 
Ceph issue 해결 사례
Ceph issue 해결 사례Ceph issue 해결 사례
Ceph issue 해결 사례
 
Wireless Cracking using Kali
Wireless Cracking using KaliWireless Cracking using Kali
Wireless Cracking using Kali
 
Secrets of Performance Tuning Java on Kubernetes
Secrets of Performance Tuning Java on KubernetesSecrets of Performance Tuning Java on Kubernetes
Secrets of Performance Tuning Java on Kubernetes
 

Similar to Tips on High Performance Server Programming

Server Tips
Server TipsServer Tips
Server Tips
liqingfang126
 
Perfmon And Profiler 101
Perfmon And Profiler 101Perfmon And Profiler 101
Perfmon And Profiler 101
Quest Software
 
Evergreen Sysadmin Survival Skills
Evergreen Sysadmin Survival SkillsEvergreen Sysadmin Survival Skills
Evergreen Sysadmin Survival Skills
Evergreen ILS
 
MySQL Tuning
MySQL TuningMySQL Tuning
MySQL Tuning
Ford AntiTrust
 
Capacity Planning for fun & profit
Capacity Planning for fun & profitCapacity Planning for fun & profit
Capacity Planning for fun & profit
Rodrigo Campos
 
Optimizing Python
Optimizing PythonOptimizing Python
Optimizing Python
AdimianBE
 
Web 2.0 Performance and Reliability: How to Run Large Web Apps
Web 2.0 Performance and Reliability: How to Run Large Web AppsWeb 2.0 Performance and Reliability: How to Run Large Web Apps
Web 2.0 Performance and Reliability: How to Run Large Web Apps
adunne
 
Ajax Tutorial
Ajax TutorialAjax Tutorial
Ajax Tutorial
oscon2007
 
Performance Whackamole (short version)
Performance Whackamole (short version)Performance Whackamole (short version)
Performance Whackamole (short version)
PostgreSQL Experts, Inc.
 
Performance
PerformancePerformance
Performance
Christophe Marchal
 
How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Ped...
How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Ped...How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Ped...
How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Ped...
Red Hat Developers
 
Benchmarks, performance, scalability, and capacity what's behind the numbers
Benchmarks, performance, scalability, and capacity what's behind the numbersBenchmarks, performance, scalability, and capacity what's behind the numbers
Benchmarks, performance, scalability, and capacity what's behind the numbers
Justin Dorfman
 
Benchmarks, performance, scalability, and capacity what s behind the numbers...
Benchmarks, performance, scalability, and capacity  what s behind the numbers...Benchmarks, performance, scalability, and capacity  what s behind the numbers...
Benchmarks, performance, scalability, and capacity what s behind the numbers...
james tong
 
Understanding and Designing Ultra low latency systems | Low Latency | Ultra L...
Understanding and Designing Ultra low latency systems | Low Latency | Ultra L...Understanding and Designing Ultra low latency systems | Low Latency | Ultra L...
Understanding and Designing Ultra low latency systems | Low Latency | Ultra L...
Anand Narayanan
 
Kernel Recipes 2016 - Speeding up development by setting up a kernel build farm
Kernel Recipes 2016 - Speeding up development by setting up a kernel build farmKernel Recipes 2016 - Speeding up development by setting up a kernel build farm
Kernel Recipes 2016 - Speeding up development by setting up a kernel build farm
Anne Nicolas
 
Using the big guns: Advanced OS performance tools for troubleshooting databas...
Using the big guns: Advanced OS performance tools for troubleshooting databas...Using the big guns: Advanced OS performance tools for troubleshooting databas...
Using the big guns: Advanced OS performance tools for troubleshooting databas...
Nikolay Savvinov
 
Lightweight Grids With Terracotta
Lightweight Grids With TerracottaLightweight Grids With Terracotta
Lightweight Grids With Terracotta
PT.JUG
 
New School Man-in-the-Middle
New School Man-in-the-MiddleNew School Man-in-the-Middle
New School Man-in-the-Middle
Tom Eston
 
Ceph Day Melbourne - Troubleshooting Ceph
Ceph Day Melbourne - Troubleshooting Ceph Ceph Day Melbourne - Troubleshooting Ceph
Ceph Day Melbourne - Troubleshooting Ceph
Ceph Community
 
Linux Performance Tools 2014
Linux Performance Tools 2014Linux Performance Tools 2014
Linux Performance Tools 2014
Brendan Gregg
 

Similar to Tips on High Performance Server Programming (20)

Server Tips
Server TipsServer Tips
Server Tips
 
Perfmon And Profiler 101
Perfmon And Profiler 101Perfmon And Profiler 101
Perfmon And Profiler 101
 
Evergreen Sysadmin Survival Skills
Evergreen Sysadmin Survival SkillsEvergreen Sysadmin Survival Skills
Evergreen Sysadmin Survival Skills
 
MySQL Tuning
MySQL TuningMySQL Tuning
MySQL Tuning
 
Capacity Planning for fun & profit
Capacity Planning for fun & profitCapacity Planning for fun & profit
Capacity Planning for fun & profit
 
Optimizing Python
Optimizing PythonOptimizing Python
Optimizing Python
 
Web 2.0 Performance and Reliability: How to Run Large Web Apps
Web 2.0 Performance and Reliability: How to Run Large Web AppsWeb 2.0 Performance and Reliability: How to Run Large Web Apps
Web 2.0 Performance and Reliability: How to Run Large Web Apps
 
Ajax Tutorial
Ajax TutorialAjax Tutorial
Ajax Tutorial
 
Performance Whackamole (short version)
Performance Whackamole (short version)Performance Whackamole (short version)
Performance Whackamole (short version)
 
Performance
PerformancePerformance
Performance
 
How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Ped...
How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Ped...How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Ped...
How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Ped...
 
Benchmarks, performance, scalability, and capacity what's behind the numbers
Benchmarks, performance, scalability, and capacity what's behind the numbersBenchmarks, performance, scalability, and capacity what's behind the numbers
Benchmarks, performance, scalability, and capacity what's behind the numbers
 
Benchmarks, performance, scalability, and capacity what s behind the numbers...
Benchmarks, performance, scalability, and capacity  what s behind the numbers...Benchmarks, performance, scalability, and capacity  what s behind the numbers...
Benchmarks, performance, scalability, and capacity what s behind the numbers...
 
Understanding and Designing Ultra low latency systems | Low Latency | Ultra L...
Understanding and Designing Ultra low latency systems | Low Latency | Ultra L...Understanding and Designing Ultra low latency systems | Low Latency | Ultra L...
Understanding and Designing Ultra low latency systems | Low Latency | Ultra L...
 
Kernel Recipes 2016 - Speeding up development by setting up a kernel build farm
Kernel Recipes 2016 - Speeding up development by setting up a kernel build farmKernel Recipes 2016 - Speeding up development by setting up a kernel build farm
Kernel Recipes 2016 - Speeding up development by setting up a kernel build farm
 
Using the big guns: Advanced OS performance tools for troubleshooting databas...
Using the big guns: Advanced OS performance tools for troubleshooting databas...Using the big guns: Advanced OS performance tools for troubleshooting databas...
Using the big guns: Advanced OS performance tools for troubleshooting databas...
 
Lightweight Grids With Terracotta
Lightweight Grids With TerracottaLightweight Grids With Terracotta
Lightweight Grids With Terracotta
 
New School Man-in-the-Middle
New School Man-in-the-MiddleNew School Man-in-the-Middle
New School Man-in-the-Middle
 
Ceph Day Melbourne - Troubleshooting Ceph
Ceph Day Melbourne - Troubleshooting Ceph Ceph Day Melbourne - Troubleshooting Ceph
Ceph Day Melbourne - Troubleshooting Ceph
 
Linux Performance Tools 2014
Linux Performance Tools 2014Linux Performance Tools 2014
Linux Performance Tools 2014
 

More from Joshua Zhu

阿里开源经验分享
阿里开源经验分享阿里开源经验分享
阿里开源经验分享
Joshua Zhu
 
阿里云CDN技术演进之路
阿里云CDN技术演进之路阿里云CDN技术演进之路
阿里云CDN技术演进之路
Joshua Zhu
 
阿里CDN技术揭秘
阿里CDN技术揭秘阿里CDN技术揭秘
阿里CDN技术揭秘
Joshua Zhu
 
Nginx深度開發與客制化
Nginx深度開發與客制化Nginx深度開發與客制化
Nginx深度開發與客制化
Joshua Zhu
 
Hacking Nginx at Taobao
Hacking Nginx at TaobaoHacking Nginx at Taobao
Hacking Nginx at Taobao
Joshua Zhu
 
Velocity 2010 Highlights
Velocity 2010 HighlightsVelocity 2010 Highlights
Velocity 2010 Highlights
Joshua Zhu
 

More from Joshua Zhu (6)

阿里开源经验分享
阿里开源经验分享阿里开源经验分享
阿里开源经验分享
 
阿里云CDN技术演进之路
阿里云CDN技术演进之路阿里云CDN技术演进之路
阿里云CDN技术演进之路
 
阿里CDN技术揭秘
阿里CDN技术揭秘阿里CDN技术揭秘
阿里CDN技术揭秘
 
Nginx深度開發與客制化
Nginx深度開發與客制化Nginx深度開發與客制化
Nginx深度開發與客制化
 
Hacking Nginx at Taobao
Hacking Nginx at TaobaoHacking Nginx at Taobao
Hacking Nginx at Taobao
 
Velocity 2010 Highlights
Velocity 2010 HighlightsVelocity 2010 Highlights
Velocity 2010 Highlights
 

Recently uploaded

Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Fwdays
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
Javier Junquera
 

Recently uploaded (20)

Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
 

Tips on High Performance Server Programming

  • 1. Tips on High Performance Server Programming Joshua Zhu June 9, 2009
  • 2. Agenda • Fundamentals • Best practices and common tricks • Design issues • Tools and resources
  • 3. Key Rules • Do NOT block • Avoid excessive system calls • Cache/preprocess as much as possible • Use efficient algorithms and data structures • Threads are usually a bad idea • Separate the I/O code from the business-logic code • Keep the most heavily used data near the CPU • Tune against the bottleneck
  • 4. I/O Models • Blocking • Non-blocking • I/O multiplexing • Signal-driven I/O • Asynchronous I/O
  • 5. I/O Multiplexing • Select • Poll • /dev/poll • Epoll/kqueue – Level triggered – Edge triggered
  • 6. I/O Strategies • 1 thread, non-blocking, level-triggered • 1 thread, non-blocking, edge-triggered • 1 thread, AIO • 1 thread per client • Build the server code into the kernel
  • 7. Let’s Face the C10K Problem • 10000 connections – CPU/memory usage per connection • 10000 hits/sec – 10 us per hit • Instruction (ns) • System call (us)
  • 8. Agenda • Fundamentals • Best practices and common tricks • Design issues • Tools and resources
  • 9. The Event-driven Model while (true) { for t in run_tasks: t.handler(); update_time(&now); timeout = ETERNITY; for t in wait_tasks: /* sorted already */ if (t.time <= now) { t.timeout_handler(); } else { timeout = t.time - now; break; } nevents = poll_function(events, timeout); for i in nevents: task t; if (events[i].type == READ) { t.handler = read_handler; } else (events[i].type == WRITE) { t.handler = write_handler; } run_tasks_add(t); }
  • 10. Scheduler • Task scheduling – Run queue – Wait queue • Timers – Time jumps – The next timeout • Pass to select()/poll()/epoll_wait()... – Data structures • Red-black tree • Min-heap • Timer wheel
  • 11. Unify Multiple Event Sources • I/O events • Timer events • Signals/threads – Use a pipe or socketpair • Register the read end in the event loop • Notify the event loop by writing to the write end
  • 12. Buffer Management • Fixed size or not • R/W pointers operation – Producer/consumer • Circular buffer • Single buffering • Buffer chain
  • 13. Memory • Memory fragment – Memory pool • Fixed size – MRU • Non-fixed size – Slab allocator • Memory operations are expensive – Allocation/free – Data copies • Pre-allocation/static-allocation – Array
  • 14. Strings • Gotchas – Strlen() • Sizeof() - 1 – Strncpy() • Strlcpy() – Vsnprintf() –… • Algorithms – KMP/BM – AC/WM –…
  • 15. Caching • Time – Reduce the number of gettimeofday() calls – Time resolution • ms or sec? • Content – Files • In-memory buffering – Database • Memcached • Preprocessing
  • 16. Accept() Strategies • Accept-limit – Multi-accept – Accept-mutex • The thundering herd problem – Polling functions – Kernels – The amount of processes/threads
  • 17. Concurrency • Threads – Pros • Utilize all the CPUs • True CPU concurrency – Cons • Context switches • Locks hurt performance – Deadlocks – Starvation – Race conditions • Error prone and hard to debug • Hard limits of operating systems
  • 18. Concurrency (cont’d) • Categorize your service – CPU-bound – I/O-bound • Models – Thread pool – Process pool – SEDA – Master/workers • CPU affinity
  • 19. Advanced I/O Functions • Gather read, scatter write – Readv/writev • Sendfile • Mmap • Splice and tee
  • 20. Take Advantage of TCP/IP Options • TCP/IP options – SO_REUSEADDR – SO_RCVBUF/SO_SNDBUF – TCP_CORK – TCP_NODELAY – TCP_DEFER_ACCEPT –…
  • 21. Misc. • Byte order – Little-endian – Big-endian • Max open file number – Ulimit/setrlimit
  • 22. Agenda • Fundamentals • Best practices and common tricks • Design issues • Tools and resources
  • 23. Protocol • Readability – Binary – Textual • Compatibility • Security – Heart beat – Invalid data • The state machine (FSM) – Granularity – Traceability
  • 24. Security • Timeouts • Bad data • Buffer overflow • DOS attack • Encryption • Privilege • Working directory
  • 25. Flexibility • Modularize – Modules – Plugins • Scriptable – Lua • Hot code update • HA
  • 26. Introspection • Statistics – SNMP • Debugable • Tunable • Logs
  • 27. Patterns • Reactor • Proactor • Half-Sync/Half-Async • Leader/Followers • …
  • 28. Choose a Suitable Language • C • C++ • Java • Erlang • …
  • 29. The Devil Hides in the Details • Find out the shortest plank in the bucket – Profile/benchmark • Take care of every line of your code! – Return values – Error code • EINTR • SIGPIPE • … • Tune your operating system if necessary – /proc • Make sure hardware is not the bottleneck – Network cards – Disk – …
  • 30. Agenda • Fundamentals • Best practices and common tricks • Design issues • Tools and resources
  • 31. Sharpen Your Tools • Ping • Traceroute • Netstat • Lsof • Strace • Nc • Wireshark • Gprof • Valgrind • Vmstat/iostat/dstat • Dot/gnuplot
  • 32. Understand TCP/IP Well • Three-way handshake • Connection tear-down • TCP state transition – TIME_WAIT – CLOSE_WAIT –…
  • 33. Learn By Reading Code • Frameworks – ACE – Libevent/libev – Boost::asio – … • Servers – HAProxy – Nginx – Lighttpd – Memcached – MySQL Proxy – …
  • 34. Recommended Books • Advanced Programming in the UNIX Environment • UNIX Network Programming • TCP/IP Illustrated • Effective TCP/IP Programming • Pattern-Oriented Software Architecture • C++ Network Programming • Programming With POSIX Threads • Introduction to Algorithms