Your SlideShare is downloading. ×
Fscons scalable appplication transfers
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Fscons scalable appplication transfers

2,273
views

Published on

Daniel Stenberg's talk at FSCONS 2010 about client-side scalable application layer transfers

Daniel Stenberg's talk at FSCONS 2010 about client-side scalable application layer transfers

Published in: Technology, Education

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,273
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Scalable application layer transfers
  • 2. Daniel Stenberg ● Free Software ● Network hacker, code and IETF ● Embedded developer ● Consultant Email: daniel@haxx.se Twitter: @bagder Web: daniel.haxx.se Blog: daniel.haxx.se/blog
  • 3. Agenda ● Traditional network client ● … applied to libcurl ● C10K ● … applied to libcurl
  • 4. Protocol stack Contents Application Transport Internet Link
  • 5. libcurl ● curl.haxx.se ● Very portable C library ● Application protocol transfers: DICT, FILE, FTP, FTPS,  GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3,  POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and  TFTP ● Roots from 1997 ● MIT licensed ● Daniel is lead developer
  • 6. A typical client ● Create socket ● Connect socket ● Select() for data to read or write ● Read or write data ● Loop
  • 7. pseudo C code WARNING the following source code  examples are not complete and  they resemble C code
  • 8. int main(int argc, char *argv[]) { int sock; /* Socket descriptor */ if ((sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError("socket() failed"); /* it would resolve the host name here */ if (connect(sock to server)) DieWithError("connect() failed"); do { select(maxfd+1, sock, …); /* wait for socket to become readable or writable */ if(check sock if writable) send(sock, string, string_len, 0); if(check sock if readable) recv(sock, string, string_len, 0); } while(everything is fine); close(sock); exit(0); } School book example 1 2 1 3 4 5
  • 9. Simple case works fine ● Easy to read ● Fast to write ● Easy to learn and teach
  • 10. int main(int argc, char *argv[]) { int sock[NUM_CONNS]; /* Socket descriptors */ for(i=0; i< NUM_CONNS; i++) { if ((sock[i] = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError("socket() failed"); if (connect(sock[i] to server[i])) DieWithError("connect() failed"); do { /* set up the bitmasks for select() */ select(maxfd+1, sock, …); /* wait for any socket to become readable or writable */ for(i=0; i< NUM_CONNS; i++) { if(check sock[i] if writable) send(sock, string, string_len, 0); if(check sock[i] if readable) recv(sock, string, string_len); } } while(everything is fine); close(sock); exit(0); } More connections 1 2 1 3 4 5
  • 11. int main(int argc, char *argv[]) { int sock[NUM_CONNS]; /* Socket descriptors */ for(i=0; i< NUM_CONNS; i++) { if ((sock[i] = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError("socket() failed"); if (connect(sock[i] to server[i])) DieWithError("connect() failed"); do { /* set up the bitmasks for select() */ select(maxfd+1, sock, …); /* wait for any socket to become readable or writable */ for(i=0; i< NUM_CONNS; i++) { if(check sock[i] if writable) send(sock, string, string_len, 0); if(check sock[i] if readable) recv(sock, string, string_len); } } while(everything is fine); close(sock); exit(0); } More connections
  • 12. Many connections ● Quickly degrades ● select() and poll() suck ● select() slightly more due to  FD_SETSIZE ● Threads help to some degree,  but...
  • 13. what about libcurl ● examples in C ● has bindings for 40 languages
  • 14. libcurl simple app int main(void) { CURL *curl; CURLcode res; curl = curl_easy_init(); if(curl) { curl_easy_setopt(curl, CURLOPT_URL, "http://example.com"); res = curl_easy_perform(curl); /* always cleanup */ curl_easy_cleanup(curl); } return 0; }
  • 15. easy interface ● easy to write ● synchronous ● supports many protocols in one  go ● made for single transfers
  • 16. libcurl multi app int main(int argc, char **argv) { handle1 = curl_easy_init(); handle2 = curl_easy_init(); curl_easy_setopt(handle1, CURLOPT_URL, "http://www.example.com/"); curl_easy_setopt(handle2, CURLOPT_URL, "http://fscons.org/"); multi_handle = curl_multi_init(); curl_multi_add_handle(multi_handle, handle1); curl_multi_add_handle(multi_handle, handle2); curl_multi_perform(multi_handle, &still_running); while(still_running) { curl_multi_timeout(multi_handle, &curl_timeo); curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd); rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout); if(rc says timeout or readable/writable sockets) curl_multi_perform(multi_handle, &still_running); } curl_multi_cleanup(multi_handle); curl_easy_cleanup(http_handle); curl_easy_cleanup(http_handle2); return 0; } 1 2 5 4 6 3 7 8
  • 17. libcurl multi app int main(int argc, char **argv) { handle1 = curl_easy_init(); handle2 = curl_easy_init(); curl_easy_setopt(handle1, CURLOPT_URL, "http://www.example.com/"); curl_easy_setopt(handle2, CURLOPT_URL, "http://fscons.org/"); multi_handle = curl_multi_init(); curl_multi_add_handle(multi_handle, handle1); curl_multi_add_handle(multi_handle, handle2); curl_multi_perform(multi_handle, &still_running); while(still_running) { curl_multi_timeout(multi_handle, &curl_timeo); curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd); rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout); if(rc says timeout or readable/writable sockets) curl_multi_perform(multi_handle, &still_running); } curl_multi_cleanup(multi_handle); curl_easy_cleanup(http_handle); curl_easy_cleanup(http_handle2); return 0; }
  • 18. multi interface ● several “easy” handles dealt with  combined ● made for simultaneous transfers ● still easy to write and read ● but: relies on select()
  • 19. select with many
  • 20. Scale to c10K ● 10000 simultaneous connections ● not as easy as it may sound
  • 21. A better way ● event­based ● react only on the sockets that  have changed actions ● no “check for action” ● no loops
  • 22. really scalable
  • 23. not that portable ● There's no POSIX for event­based ● The portable libs don't support all  platforms ● epoll (Linux), kqueue (FreeBSD, NetBSD,  OpenBSD, Darwin), /dev/poll (Solaris, HPUX),  pollset (AIX), Event Completion (Solaris 10), I/O  Completion Ports (Microsoft Windows)
  • 24. libev and libevent ● potent event libraries we base the  examples on in this presentation ● they support a subset of platforms ● the examples are not complete now  either
  • 25. event­based basics ● detail what sockets we work with  and what do we wait for on the  sockets ● timeouts ● wait until something happens
  • 26. simple event client int function(int socketfd) { ev = ev_default_loop(); ev_timer_init(..., timer_callback, …); ev_io_init(ev, event_callback, socketfd, WHAT); ev_loop(ev); /* sit here until done */ } int timer_callback(...) { /* timer expired! */ } int event_callback(..., int socketfd, ...) { /* something happened on my socket, now we recv or send etc */ }
  • 27. event based compared ● different code­flow ● might require a different state  machine for the protocol ● again: not widely portable
  • 28. Other ways to avoid  wrongs ● one thread per connection with  blocking sockets ● threads with non­blocking i/o to  handle N connections each
  • 29. using libcurl scalable ● “multi_socket” API added 2006 ● tells the app what sockets to wait for  what on ● app then speaks with event library ● avoids select() ● event system agnostic
  • 30. libcurl multi_socket  bindings ● not always supported ● a matter of writing the glue code  correct – you can help!
  • 31. multi_socket example int main() { CURLM *multi = curl_multi_init(); CURL *easy; curl_multi_setopt(..., CURLMOPT_SOCKETFUNCTION, socket_cb); curl_multi_setopt(..., CURLMOPT_TIMERFUNCTION, timer_cb); easy = curl_easy_init(); curl_easy_setopt(..., CURLOPT_URL, …); curl_multi_add_handle(..., easy); event_dispatch(); /* start the libevent dispatch loop */ } int event_cb(...) { /* event library found event on libcurl's socket, act */ curl_multi_socket_action(..., action, ...); } int socket_cb(...) { /* called when libcurl adds, removes or changes a socket that we must deal with */ event_set(..., event_cb); /* tell event lib about setup */ }
  • 32. Scales? ● done 70,000+ connections ● port numbers are 16 bit ● callbacks take time thus limit  throughput
  • 33. Which protocols? ● (almost) all network based ones  libcurl support! (DICT, FTP, FTPS, GOPHER,  HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3,  POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS and  TFTP)
  • 34. Caveats? ● yeah sure, but... ● c­ares for name resolving ● not everything is non­blocking
  • 35. More factors ● Network stack inefficiency ● Stack size? ● Memory / mallocs ● “Other” code may slow down things ● Copy copy copy copy ...
  • 36. For you who fell asleep ● old style socket applications  don't scale ● use event based concepts ● libcurl can too
  • 37. libcurl is... ● entirely open source ● in need of your help! ● at http://curl.haxx.se/libcurl/