Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Everything you wanted to know about writing async, concurrent http apps in java

5,062 views

Published on

As presented at CodeMotion Tel Aviv:

Facing tens of millions of clients continuously downloading binaries from its repositories, JFrog decided to offer an OSS client that natively supports these downloads. This session shares the main challenges of developing a highly concurrent, resumable, async download library on top of an Apache HTTP client. It also covers other libraries JFrog tested and why it decided to reinvent the wheel. Consider yourself forewarned: lots of HTTP internals, NIO, and concurrency ahead!

Published in: Software
  • Sex in your area is here: ♥♥♥ http://bit.ly/39pMlLF ♥♥♥
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ♥♥♥ http://bit.ly/39pMlLF ♥♥♥
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Everything you wanted to know about writing async, concurrent http apps in java

  1. 1. Everything You Wanted to Know About Writing Async, Concurrent HTTP Apps in Java
  2. 2. Agenda • Mostly this:
  3. 3. Agenda • And this:
  4. 4. Agenda • And this:
  5. 5. About your speaker linkd.in/jbaruch stackoverflow.com/users/402053/jbaruch
  6. 6. What Frog?
  7. 7. What Frog?
  8. 8. What Frog?
  9. 9. What Frog?
  10. 10. Requirements – parallel file Downloads – Parallel file parts – interrupt/pause/resume – Progress events – Checksums caching
  11. 11. First Association for “concurrent downloader”
  12. 12. Lucky day: Download manager written in java!
  13. 13. Let’s look if we can use it! 1. No traceable license 2. No website or docs 3. No traceable sources 4. It’s an app, not a lib
  14. 14. Java.net.urlconnection 1. Memory wasteful (buffering) 2. Minimal API 3. Blocking streams
  15. 15. What we’re looking for 1. Async/non-blocking 2. Event callbacks
  16. 16. What is IT going to take 1. Reactor 2. nio
  17. 17. Welcome to the reactor
  18. 18. – pattern for lightweight concurrency – Event driven – Threads reuse – Uses non-blocking Io
  19. 19. Original pattern http://www.dre.vanderbilt.edu/~schmidt/PDF/reactor-siemens.pdf
  20. 20. Guess the author by the diagram http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf
  21. 21. In Java, Reactor means NIO
  22. 22. Selector as a multiplexer
  23. 23. Java version - Registering SocketChannel channel= SocketChannel.open(); socketChannel.connect(new InetSocketAddress("http://remote.com", 80)); ... Selector selector = Selector.open(); channel.configureBlocking(false); SelectionKey k = channel.register(selector, SelectionKey.OP_READ); k.attach(handler);
  24. 24. Java version - Dispatcher while (!Thread.interrupted()) { selector.select(); Set selected = selector.selectedKeys(); Iterator it = selected.iterator(); while (it.hasNext()) SelectionKey k = (SelectionKey)(it.next(); ((Runnable)(k.attachment())).run(); selected.clear(); }
  25. 25. Handling reactor events is complex – Need to maintain state – Buffering – assembling chunks – Coordinating async events
  26. 26. Nio libraries – Most of them are servers –Netty, grizzly, etc. – Apache Mina – Apache HTTP components asyncclient – Ning http client
  27. 27. – Client and server nio library – Evolved from netty – Latest release October 2012
  28. 28. Nio libraries – Most of them are servers –Netty, grizzly, etc – Apache Mina – Apache HTTP components asyncclient – Ning http client
  29. 29. Ning’s async http client
  30. 30. Here it is!
  31. 31. try (AsyncHttpClient asyncHttpClient = new AsyncHttpClient()) { ListenableFuture<Response> future = asyncHttpClient.prepareGet( "http://oss.jfrog.org/api/system/ping").execute( new AsyncCompletionHandler<Response>() { @Override public Response onCompleted(Response response) { System.out.println(response.getResponseBody()); return response; } @Override public void onThrowable(Throwable t) { t.printStackTrace(); } }); Response response = future.get(); }
  32. 32. HAC Concepts – Request producer – Response consumer
  33. 33. try (CloseableHttpAsyncClient asyncHttpClient = HttpAsyncClients.createDefault()) { asyncHttpClient.start(); Future<HttpResponse> future = asyncHttpClient.execute( HttpAsyncMethods.createGet("http://oss.jfrog.org/api/system/ping"), new AsyncByteConsumer<HttpResponse>() { @Override protected void onResponseReceived(final HttpResponse response) { System.out.println(response.getStatusLine().getReasonPhrase()); } @Override protected void onByteReceived(final CharBuffer buf, final IOControl ioctrl) { } @Override protected void releaseResources() { } @Override protected HttpResponse buildResult(final HttpContext context) { return (HttpResponse) context.getAttribute("http.response"); } }, null); HttpResponse response = future.get(); }
  34. 34. Choosing between ning and http asyncclient
  35. 35. "All problems in computer science can be solved by another level of indirection" David Wheeler
  36. 36. public interface HttpProviderDownloadHandler { void onResponseReceived(int statusCode, Map<String, List<String>> headers); boolean onBytesReceived(ByteBuffer buf); void onFailed(Throwable error); void onCanceled(); void onCompleted(); }
  37. 37. Head to head Feature/Library Ning client Http Async Client Maturity Good Very new (early 2014) Download cancelation Easy With bugs Progress hooks Events not granular enough Just use onByteReceived() Documentation A bit sparse Minimal Server-side counterpart None, client only org.apache.httpcomponents httpcore-nio
  38. 38. Performance? 900 800 700 600 500 400 300 200 100 0 Small file Medium file Large file Ning AHAC http://blogs.atlassian.com/2013/07/http-client-performance-io/
  39. 39. Rfc2616: a universe of its own
  40. 40. Confused?
  41. 41. Just read some stackoverflow (and improve your rep as you go)
  42. 42. And that one for discovering that range header is lost on redirect
  43. 43. Question! What should be content-length when using compression?
  44. 44. https://github.com/http2/http2-spec/issues/46
  45. 45. Question! Why when redirected to CDN all the chunks start from zero?
  46. 46. HttpAsyncClientBuilder builder = HttpAsyncClients.custom(); // add redirect strategy that copies "range" headers, if exist builder.setRedirectStrategy(new DefaultRedirectStrategy() { @Override public HttpUriRequest getRedirect(HttpRequest request, HttpResponse response, HttpContext context) HttpUriRequest redirectRequest = super.getRedirect(request, response, context); // copy "Range" headers, if exist Header[] rangeHeaders = request.getHeaders(HttpHeaders.RANGE); if (rangeHeaders != null) { for (Header header : rangeHeaders) { redirectRequest.addHeader(header); } } return redirectRequest; }});
  47. 47. Question! How many simultaneous connections should I open?
  48. 48. Question! What’s wrong with the following code?
  49. 49. public static String encodeUrl(String urlStr) { URLEncoder.encode(urlStr, "UTF-8"); ... }
  50. 50. Decoded URLs cannot be re-encoded to the same form http://example.com/?query=a&b==c Cannot be decoded back after it was encoded: http://example.com/?query=a%26b==c
  51. 51. Don’t use java.net.URLEncoder “Utility class for HTML form encoding. This class contains static methods for converting a String to the application/x-www-form-urlencoded MIME format. For more information about HTML form encoding, consult the HTML specification.”
  52. 52. AHC Alternatives
  53. 53. Question! How do I close a socket correctly?
  54. 54. How hard can it be to close a socket?
  55. 55. The art of socket closing http://www.safaribooksonline.com/library/view/http-the-definitive/1565925092/ch04s07.html
  56. 56. Half-closed: no new customers
  57. 57. Never block in socket close() • The other side expects you to clean up nicely • It will give up on time out • You will wait (forever)
  58. 58. Remember?
  59. 59. Question! How can I write file parts concurrently?
  60. 60. – Write to separate files, combine on finish – Write to same file, seeking to the right position
  61. 61. USE FileChannel • Implements SeekableByteChannel java.nio.channels.FileChannel#write( java.nio.ByteBuffer src, long position)
  62. 62. download progress tracking * FileProgressInfo FilePartProgressInfo • PersistentFileProgressInfo – Save the total size, sha1, number of parts – State of each part (offset, size, completed...)
  63. 63. File Locking
  64. 64. File locking Levels – VM level – OS level
  65. 65. OS level File locking • Multiple downloader instances writing to the same file • Needed for writing: – Partial download file – Persistent download progress
  66. 66. FileLock lock = fileChannel.tryLock(); //Non-shared: (0L, Long.MAX_VALUE, false) if (lock == null) { throw new OverlappingFileLockException(); } return lock; } OS Level File Locking - Exclusive
  67. 67. private FileLock lock(FileChannel fileChannel) throws IOException { FileLock lock = fileChannel.tryLock(Long.MAX_VALUE - 1, 1, false); if (lock == null) { throw new OverlappingFileLockException(); } return lock; } OS Level File Locking – Advisory exclusive WTF?!
  68. 68. VM Level File Locking
  69. 69. VM Level File Locking – Prevent same VM threads writing to the file when we started closing it – Closing sequence: – Release file locks – Close channels – Rename a file to it's final name (remove .part) – Erase progress info
  70. 70. VM Level File Locking ReentrantReadWriteLock.ReadLock writeToFileLock = rwl.readLock(); ReentrantReadWriteLock.WriteLock closeFileLock = rwl.writeLock(); public void close() throws IOException { this.closeFileLock.lock(); } public int write(int partIndex, ByteBuffer buf) { if (!this.writeToFileLock.tryLock()) { throw new IllegalStateException("File is being closed"); } ... }
  71. 71. What’s next?
  72. 72. http/2 – Mostly standardizing Google's spdy – Header compression – multiplexing – Prioritization – Server push – On the way clear some stuff – E.g. compressed content length
  73. 73. Ease the load
  74. 74. Links! • RTFM: RFC 2616 • Ultimate book: HTTP: The Definitive Guide – Amazon – Safari • Reactor pattern • Doug Lea on NIO
  75. 75. No, Thank you!

×