Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Fscons scalable appplication transfers

on

  • 2,435 views

Daniel Stenberg's talk at FSCONS 2010 about client-side scalable application layer transfers

Daniel Stenberg's talk at FSCONS 2010 about client-side scalable application layer transfers

Statistics

Views

Total Views
2,435
Views on SlideShare
2,326
Embed Views
109

Actions

Likes
1
Downloads
4
Comments
0

3 Embeds 109

http://daniel.haxx.se 91
http://www.advogato.org 17
http://static.slidesharecdn.com 1

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Fscons scalable appplication transfers Fscons scalable appplication transfers Presentation Transcript

  • Scalable application layer transfers
  • Daniel Stenberg
    • Free Software
    • Network hacker, code and IETF
    • Embedded developer
    • Consultant
    Email: [email_address] Twitter: @bagder Web: daniel.haxx.se Blog: daniel.haxx.se/blog
  • Agenda
    • Traditional network client
    • … applied to libcurl
    • C10K
    • … applied to libcurl
  • Protocol stack Contents Application Transport Internet Link
  • libcurl
    • curl.haxx.se
    • Very portable C library
    • Application protocol transfers: DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP
    • Roots from 1997
    • MIT licensed
    • Daniel is lead developer
  • A typical client
    • Create socket
    • Connect socket
    • Select() for data to read or write
    • Read or write data
    • Loop
  • pseudo C code
      WARNING the following source code examples are not complete and they resemble C code
  • int main(int argc, char *argv[]) { int sock; /* Socket descriptor */ if ((sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError(&quot;socket() failed&quot;); /* it would resolve the host name here */ if (connect(sock to server)) DieWithError(&quot;connect() failed&quot;); do { select(maxfd+1, sock, …); /* wait for socket to become readable or writable */ if(check sock if writable) send(sock, string, string_len, 0); if(check sock if readable) recv(sock, string, string_len, 0); } while(everything is fine); close(sock); exit(0); } School book example 1 2 1 3 4 5
  • Simple case works fine
    • Easy to read
    • Fast to write
    • Easy to learn and teach
  • int main(int argc, char *argv[]) { int sock[NUM_CONNS]; /* Socket descriptors */ for(i=0; i< NUM_CONNS; i++) { if ((sock[i] = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError(&quot;socket() failed&quot;); if (connect(sock[i] to server[i])) DieWithError(&quot;connect() failed&quot;); do { /* set up the bitmasks for select() */ select(maxfd+1, sock, …); /* wait for any socket to become readable or writable */ for(i=0; i< NUM_CONNS; i++) { if(check sock[i] if writable) send(sock, string, string_len, 0); if(check sock[i] if readable) recv(sock, string, string_len); } } while(everything is fine); close(sock); exit(0); } More connections 1 2 1 3 4 5
  • int main(int argc, char *argv[]) { int sock[NUM_CONNS]; /* Socket descriptors */ for(i=0; i< NUM_CONNS; i++) { if ((sock[i] = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError(&quot;socket() failed&quot;); if (connect(sock[i] to server[i])) DieWithError(&quot;connect() failed&quot;); do { /* set up the bitmasks for select() */ select(maxfd+1, sock, …); /* wait for any socket to become readable or writable */ for(i=0; i< NUM_CONNS; i++) { if(check sock[i] if writable) send(sock, string, string_len, 0); if(check sock[i] if readable) recv(sock, string, string_len); } } while(everything is fine); close(sock); exit(0); } More connections
  • Many connections
    • Quickly degrades
    • select() and poll() suck
    • select() slightly more due to FD_SETSIZE
    • Threads help to some degree, but...
  • what about libcurl
    • examples in C
    • has bindings for 40 languages
  • libcurl simple app int main(void) { CURL *curl; CURLcode res; curl = curl_easy_init(); if(curl) { curl_easy_setopt(curl, CURLOPT_URL, &quot;http://example.com&quot;); res = curl_easy_perform(curl); /* always cleanup */ curl_easy_cleanup(curl); } return 0; }
  • easy interface
    • easy to write
    • synchronous
    • supports many protocols in one go
    • made for single transfers
  • libcurl multi app int main(int argc, char **argv) { handle1 = curl_easy_init(); handle2 = curl_easy_init(); curl_easy_setopt(handle1, CURLOPT_URL, &quot;http://www.example.com/&quot;); curl_easy_setopt(handle2, CURLOPT_URL, &quot;http://fscons.org/&quot;); multi_handle = curl_multi_init(); curl_multi_add_handle(multi_handle, handle1); curl_multi_add_handle(multi_handle, handle2); curl_multi_perform(multi_handle, &still_running); while(still_running) { curl_multi_timeout(multi_handle, &curl_timeo); curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd); rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout); if(rc says timeout or readable/writable sockets) curl_multi_perform(multi_handle, &still_running); } curl_multi_cleanup(multi_handle); curl_easy_cleanup(http_handle); curl_easy_cleanup(http_handle2); return 0; } 1 2 5 4 6 3 7 8
  • libcurl multi app int main(int argc, char **argv) { handle1 = curl_easy_init(); handle2 = curl_easy_init(); curl_easy_setopt(handle1, CURLOPT_URL, &quot;http://www.example.com/&quot;); curl_easy_setopt(handle2, CURLOPT_URL, &quot;http://fscons.org/&quot;); multi_handle = curl_multi_init(); curl_multi_add_handle(multi_handle, handle1); curl_multi_add_handle(multi_handle, handle2); curl_multi_perform(multi_handle, &still_running); while(still_running) { curl_multi_timeout(multi_handle, &curl_timeo); curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd); rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout); if(rc says timeout or readable/writable sockets) curl_multi_perform(multi_handle, &still_running); } curl_multi_cleanup(multi_handle); curl_easy_cleanup(http_handle); curl_easy_cleanup(http_handle2); return 0; }
  • multi interface
    • several “easy” handles dealt with combined
    • made for simultaneous transfers
    • still easy to write and read
    • but: relies on select()
  • select with many
  • Scale to c10K
    • 10000 simultaneous connections
    • not as easy as it may sound
  • A better way
    • event-based
    • react only on the sockets that have changed actions
    • no “check for action”
    • no loops
  • really scalable
  • not that portable
    • There's no POSIX for event-based
    • The portable libs don't support all platforms
    • epoll (Linux), kqueue (FreeBSD, NetBSD, OpenBSD, Darwin), /dev/poll (Solaris, HPUX), pollset (AIX), Event Completion (Solaris 10), I/O Completion Ports (Microsoft Windows)
  • libev and libevent
    • potent event libraries we base the examples on in this presentation
    • they support a subset of platforms
    • the examples are not complete now either
  • event-based basics
    • detail what sockets we work with and what do we wait for on the sockets
    • timeouts
    • wait until something happens
  • simple event client int function(int socketfd) { ev = ev_default_loop(); ev_timer_init(..., timer_callback, …); ev_io_init(ev, event_callback, socketfd, WHAT); ev_loop(ev); /* sit here until done */ } int timer_callback(...) { /* timer expired! */ } int event_callback(..., int socketfd, ...) { /* something happened on my socket, now we recv or send etc */ }
  • event based compared
    • different code-flow
    • might require a different state machine for the protocol
    • again: not widely portable
  • Other ways to avoid wrongs
    • one thread per connection with blocking sockets
    • threads with non-blocking i/o to handle N connections each
  • using libcurl scalable
    • “multi_socket” API added 2006
    • tells the app what sockets to wait for what on
    • app then speaks with event library
    • avoids select()
    • event system agnostic
  • libcurl multi_socket bindings
    • not always supported
    • a matter of writing the glue code correct – you can help!
  • multi_socket example int main() { CURLM *multi = curl_multi_init(); CURL *easy; curl_multi_setopt(..., CURLMOPT_SOCKETFUNCTION, socket_cb); curl_multi_setopt(..., CURLMOPT_TIMERFUNCTION, timer_cb); easy = curl_easy_init(); curl_easy_setopt(..., CURLOPT_URL, …); curl_multi_add_handle(..., easy); event_dispatch(); /* start the libevent dispatch loop */ } int event_cb(...) { /* event library found event on libcurl's socket, act */ curl_multi_socket_action(..., action, ...); } int socket_cb(...) { /* called when libcurl adds, removes or changes a socket that we must deal with */ event_set(..., event_cb); /* tell event lib about setup */ }
  • Scales?
    • done 70,000+ connections
    • port numbers are 16 bit
    • callbacks take time thus limit throughput
  • Which protocols?
    • (almost) all network based ones libcurl support! (DICT, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS and TFTP)
  • Caveats?
    • yeah sure, but...
    • c-ares for name resolving
    • not everything is non-blocking
  • More factors
    • Network stack inefficiency
    • Stack size?
    • Memory / mallocs
    • “Other” code may slow down things
    • Copy copy copy copy ...
  • For you who fell asleep
    • old style socket applications don't scale
    • use event based concepts
    • libcurl can too
  • libcurl is...
    • entirely open source
    • in need of your help!
    • at http://curl.haxx.se/libcurl/