Successfully reported this slideshow.

Introduction httpClient on Java11 / Java11時代のHTTPアクセス再入門

9

Share

Loading in …3
×
1 of 86
1 of 86

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Introduction httpClient on Java11 / Java11時代のHTTPアクセス再入門

  1. 1. Java11時代のHTTPアクセス再入門 JJUG CCC 2019 Spring @tamtam180 2019/05/18
  2. 2. アジェンダ • 自己紹介 • おさらい – HttpUrlConnectionの話 – Apache HttpClientの話 • java.net.http.HttpClientの話
  3. 3. アジェンダ • 話さないこと – HTTP/2の基本的な話
  4. 4. 自己紹介 • Name: Kiyotaka Suzuki • Twitter: @tamtam180 • Main works – Square Enix (5.5 Year) • PlayOnline, FF-XIV, etc, – SmartNews (5 Year) • Cross functional Expert Team • Senior Software Engineer • Advertising System, PoC, SRE, etc,.. • I Love OSS about Datastore • Mad performance tuner
  5. 5. その前に • 検証に便利なWebサイト – https://http2.pro/api/v1 • HTTP/2のテスト – https://httpbin.org/ • HTTPメソッド、ステータス、Cookie、Cache、etc • このサイトでほぼ事足りる – https://httpstat.us/ • HTTPステータス – https://badssl.com/ • HTTPSのエラーテスト – https://www.websocket.org/echo.html • WebSocketのテスト
  6. 6. おさらい • 今までの主なHTTPアクセス – HttpUrlConnection – Apache HttpClient – OkHttp
  7. 7. おさらい • HttpUrlConnection – 同期処理だけ – HTTP1.1まで – Cookieは一応扱える • OnMemory – Basic認証も出来る • QueryStringを作るのが大変(個人的感想) • Responseのgzip処理とか大変
  8. 8. おさらい • Apache HTTPClient – 5系からHTTP/2が使える(5系はβバージョン) – 3系から4系でプログラムの構造が大きく変更 • Commons HttpClientは3系, もう古い – IO Model • Blocking I/O – Classic Java IO – Non blocking • Event Driven I/O with JavaNIO
  9. 9. おさらい • Apache HTTPClient – Core • v4.4 • v5.0 beta – Client • v4.5 • v5.0 beta – HttpAsyncClient • v4.1
  10. 10. おさらい • OkHttp (v3.14.1) – HTTP/2対応 – Androidも対応 • Android5.0+ (API Level 21+) – Java8+ – TLS1.3対応
  11. 11. おさらい • java.net.http / HttpClient – HTTP/2対応 – WebSocket対応 – 同期/非同期 処理対応 – Reactive Streamとして Request/Responseを処理 – Builderパターン
  12. 12. おさらい HttpUrlConnection
  13. 13. おさらい: HttpUrlConnection • GET var url = new URL("https://httpbin.org/get"); var conn = (HttpURLConnection) url.openConnection(); try (InputStream in = conn.getInputStream()) { System.out.println(conn.getResponseCode()); System.out.println(IOUtils.toString(in, StandardCharsets.UTF_8)); } conn.disconnect(); "headers": { "Accept": "text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2", "Host": "httpbin.org", "User-Agent": "Java/11.0.2" }
  14. 14. おさらい: HttpUrlConnection • メソッドを変更する(例えばHEAD) conn.setRequestMethod("HEAD");
  15. 15. おさらい: HttpUrlConnection • Request Headerを弄る conn.setRequestProperty("User-Agent", "MyJava/11.0.0"); conn.setRequestProperty("Content-Type", "application/json");
  16. 16. おさらい: HttpUrlConnection • BODYを送信する conn.setDoOutput(true); try (var writer = new OutputStreamWriter(conn.getOutputStream())) { writer.write(new Gson().toJson(Map.of("hello", "world"))); writer.flush(); }
  17. 17. おさらい: HttpUrlConnection • POST(組み合わせ) var url = new URL("http://httpbin.org/post"); HttpURLConnection conn = (HttpURLConnection) url.openConnection(); conn.setDoOutput(true); conn.setRequestMethod("POST"); conn.setRequestProperty("User-Agent", "MyJava/11.0.0"); conn.setRequestProperty("Content-Type", "application/json"); try (OutputStreamWriter writer = new OutputStreamWriter(conn.getOutputStream())) { writer.write(new Gson().toJson(Map.of("hello", "world"))); writer.flush(); } try (InputStream in = conn.getInputStream()) { System.out.println(conn.getResponseCode()); System.out.println(IOUtils.toString(in, StandardCharsets.UTF_8)); } conn.disconnect();
  18. 18. おさらい: HttpUrlConnection • Cookie – CookieManager, CookieHandlerを使う – 世の中にあるサンプル、本当に動く?? – CookieManagerはThreadSafeじゃない CookieManager cookieManager = new CookieManager(null, CookiePolicy.ACCEPT_ALL); CookieHandler.setDefault(cookieManager); System.out.println(conn.getResponseCode()); CookieHandler.getDefault().put(conn.getURL().toURI(), conn.getHeaderFields()); for (HttpCookie cookie : cookieManager.getCookieStore().get(conn.getURL().toURI())) { System.out.println(cookie); } これが無いと動かないのだけど..
  19. 19. おさらい: HttpUrlConnection • Proxy var proxy = new Proxy( Proxy.Type.SOCKS, InetSocketAddress.createUnresolved("127.0.0.1", 7777)); HttpURLConnection conn = (HttpURLConnection) url.openConnection(proxy); ssh any-server –D7777 HTTP Proxyの場合は、 Proxy.Type.HTTP
  20. 20. おさらい: HttpUrlConnection • Redirect – デフォルトはredirectを追従 var url = new URL("https://httpbin.org/redirect/2"); var conn = (HttpURLConnection) url.openConnection(); conn.setInstanceFollowRedirects(true);
  21. 21. おさらい: HttpUrlConnection • Basic認証 – Header直接指定 – java.net.Authenticator • requestPasswordAuthenticationInstanceを 上書き • getPasswordAuthenticationを上書き
  22. 22. おさらい: HttpUrlConnection • Basic認証 – Header直接指定 var url = new URL("https://httpbin.org/basic-auth/test-user/test-pass"); var conn = (HttpURLConnection) url.openConnection(); conn.setRequestProperty( "Authorization", "Basic " + Base64.getEncoder().encodeToString("test-user:test- pass".getBytes()));
  23. 23. おさらい: HttpUrlConnection • Basic認証 – java.net.Authenticator その1 Authenticator.setDefault(new Authenticator() { @Override protected PasswordAuthentication getPasswordAuthentication() { if (getRequestorType() != RequestorType.SERVER) return null; if (!"basic".equals(getRequestingScheme())) return null; if ("httpbin.org".equalsIgnoreCase(getRequestingHost())) { return new PasswordAuthentication("test-user", "test-pass".toCharArray()); } return null; } });
  24. 24. おさらい: HttpUrlConnection • Basic認証 – java.net.Authenticator その2 Authenticator.setDefault(new Authenticator() { @Override public PasswordAuthentication requestPasswordAuthenticationInstance(String host, InetAddress addr, int port, String protocol, String prompt, String scheme, URL url, RequestorType reqType) { if (reqType != RequestorType.SERVER) return null; if (!"basic".equals(scheme)) return null; if ("httpbin.org".equalsIgnoreCase(host)) { return new PasswordAuthentication("test-user", "test-pass".toCharArray()); } return null; } });
  25. 25. おさらい: HttpUrlConnection • SSL Errorを無視する通信 SSLContext sslcontext = SSLContext.getInstance("SSL"); sslcontext.init(null, new TrustManager[]{new X509TrustManager() { public void checkClientTrusted(X509Certificate[] x509Certificates, String s) {} public void checkServerTrusted(X509Certificate[] x509Certificates, String s) {} public X509Certificate[] getAcceptedIssuers() { return null; } }}, null); HttpsURLConnection.setDefaultHostnameVerifier((s, sslSession) -> true); var url = new URL("https://expired.badssl.com/"); HttpURLConnection conn = (HttpURLConnection) url.openConnection(); if (conn instanceof HttpsURLConnection) { ((HttpsURLConnection) conn).setSSLSocketFactory(sslcontext.getSocketFactory()); } SSLContext Verifier SocketFactory
  26. 26. おさらい: HttpUrlConnection • デバッグの方法 – Logger指定 – java.net.debug
  27. 27. おさらい: HttpUrlConnection • Logger指定 [logging.properties] handlers = java.util.logging.ConsoleHandler java.util.logging.ConsoleHandler.level = ALL sun.net.www.protocol.http.HttpURLConnection.level = ALL System.setProperty("java.util.logging.config.file", "logging.properties");
  28. 28. おさらい: HttpUrlConnection • javax.net.debug – Helpを指定するとhelpが表示される System.setProperty("javax.net.debug", ”help"); URL url = new URL("https://www.google.com"); var conn = (HttpURLConnection) url.openConnection(); conn.getInputStream(); https://docs.oracle.com/javase/jp/8/docs/technotes/guides/security/jsse/JSSERefGuide.html https://www.ibm.com/support/knowledgecenter/en/SSYKE2_7.0.0/com.ibm.java.security.component.70.doc/security- all ssl:record,handshake,keygen,session,defaultctx,sslctx,sessioncache,keymanager,trustmanager,plugga bility handshake:data,verbose plaintext,packet 指定可能な値 例: ssl:handshake,session 例: all カンマ区切りかコロン区切り
  29. 29. Apache HttpClient
  30. 30. ApacheHC/GET • GET var clientBuilder = HttpClientBuilder.create(); try (CloseableHttpClient client = clientBuilder.build()) { var getMethod = new HttpGet(URI.create("http://httpbin.org/get")); try (CloseableHttpResponse resp = client.execute(getMethod)) { String body = EntityUtils.toString(resp.getEntity(), StandardCharsets.UTF_8); System.out.println(resp.getStatusLine()); System.out.println(body); } }
  31. 31. ApacheHC/Accept Header • 注意点 – デフォルトのヘッダはAccept Header無し Host: httpbin.org Connection: Keep-Alive User-Agent: Apache-HttpClient/4.5.4 (Java/11.0.1) Accept-Encoding: gzip,deflate
  32. 32. ApacheHC/Accept Header • Accept Header • (指定が無い場合は何でも受け付ける事を意 味する) • と、書いてあるのに.. RFC7231: Section 5.3.2 A request without any Accept header field implies that the user agent will accept any media type in response. https://tools.ietf.org/html/rfc7231#section-5.3.2
  33. 33. ApacheHC/Accept Header • サイトによっては正しく返却されない – 例: http://httpstat.us/200 curl -H "Accept: */*" http://httpstat.us/200 # Content-Length: 6 200 OK curl -H "Accept:" http://httpstat.us/200 # Content-Length: 0
  34. 34. ApacheHC/Accept Header • 各ブラウザのAccept Header – デフォルト値の一覧は以下のサイトを参照 https://developer.mozilla.org/en- US/docs/Web/HTTP/Content_negotiation/List_of_default_Accept_values
  35. 35. ApacheHC/Gzip • Gzipはデフォルトで処理される – gzipもdeflateも curl -s http://httpbin.org/gzip | gzip -d var clientBuilder = HttpClientBuilder.create(); try (CloseableHttpClient client = clientBuilder.build()) { var getMethod = new HttpGet("http://httpbin.org/gzip"); try (var resp = client.execute(getMethod)) { String body = EntityUtils.toString(resp.getEntity(), StandardCharsets.UTF_8); System.out.println(body); } } 本当はAccept Encodingを付ける必要がある。 httpbin.org/gzip や httpbin.org/deflate は それを無視して送ってくるのでコードでは省略している。
  36. 36. ApacheHC • Queryを作る時は URI Builder var uri = new URIBuilder("http://httpbin.org/get") .setCharset(StandardCharsets.UTF_8) .addParameter("hello", "world") .addParameter("sushi", "寿司") .setScheme("https") // HTTPSに変更 .setFragment("hello") .build(); https://httpbin.org/get?hello=world&sushi=%E5%AF%BF%E5%8F%B8#hello
  37. 37. ApacheHC • Connection Pool var connManager = new PoolingHttpClientConnectionManager(); connManager.setMaxTotal(100); connManager.setDefaultMaxPerRoute(100); var clientBuilder = HttpClientBuilder.create(); clientBuilder.setConnectionManager(connManager);
  38. 38. ApacheHC • Request Headerをいじる var clientBuilder = HttpClientBuilder.create(); var defaultHeaders = Arrays.asList( new BasicHeader("Accept", "*/*") ); clientBuilder.setDefaultHeaders(defaultHeaders); var getMethod = new HttpGet(URI.create("http://httpbin.org/get")); getMethod.setHeader("Accept", "*/*"); 定数クラスがあるのでそれを使うと良い HttpHeaders, ContentType
  39. 39. ApacheHC/POST var clientBuilder = HttpClientBuilder.create(); try (CloseableHttpClient client = clientBuilder.build()) { String jsonText = new Gson().toJson(Map.of("hello", "world")); HttpEntity entity = EntityBuilder.create() .setContentType(ContentType.APPLICATION_JSON) .setText(jsonText).build(); HttpPost postMethod = new HttpPost(URI.create("http://httpbin.org/post")); postMethod.setEntity(entity); try (CloseableHttpResponse resp = client.execute(postMethod)) { String body = EntityUtils.toString(resp.getEntity(), StandardCharsets.UTF_8); System.out.println(resp.getStatusLine()); System.out.println(body); } }
  40. 40. ApacheHC • やり方色々 – Client • HttpClientBuilder.create().build(); • HttpClients.custom().build(); • HttpClients.createDefault(); – Entity • EntityBuilder • 直接生成 BasicHttpEntity BufferedHttpEntity ByteArrayEntity FileEntity InputStreamEntity SerializableEntity StringEntity
  41. 41. ApacheHC • やり方色々 – execute • 単純に実行 • HttpContext • ResponseHandler
  42. 42. ApacheHC • Responseの処理はEntityUtilsが便利 – toString() – toByteArray() – writeTo() – consume() – ...
  43. 43. ApacheHC/Redirect • デフォルトで追従 • OFFにする場合は、RequestConfig var requestConfig = RequestConfig.custom() .setRedirectsEnabled(false) //追従しない .build(); var clientBuilder = HttpClientBuilder.create(); clientBuilder.setDefaultRequestConfig(requestConfig); var getMethod = new HttpGet(URI.create("http://httpbin.org/redirect/3")); getMethod.setConfig(requestConfig);
  44. 44. ApacheHC/Retry • DefaultHttpRequestRetryHandler – ConnectTimeoutExceptionはRetryしない – InterruptedIOExceptionのSubClassだけど https://hc.apache.org/httpcomponents-client- ga/httpclient/apidocs/org/apache/http/impl/client/DefaultHttpRequestRetryHandler.html
  45. 45. ApacheHC/Cookie var cookieStore = new BasicCookieStore(); var clientBuilder = HttpClientBuilder.create(); clientBuilder.setDefaultCookieStore(cookieStore); // return Set-Cookie try (CloseableHttpClient client = clientBuilder.build()) { var get = new HttpGet("https://httpbin.org/cookies/set/hello/world"); try (CloseableHttpResponse resp = client.execute(get)) {} } // send Cookie: hello and sushi var cookie = new BasicClientCookie("sushi", " 🍣"); cookie.setDomain("httpbin.org"); cookieStore.addCookie(cookie); try (CloseableHttpClient client = clientBuilder.build()) { var get = new HttpGet("https://httpbin.org/cookies"); try (CloseableHttpResponse resp = client.execute(get)) {} } GET /cookies HTTP/1.1 Host: httpbin.org Connection: Keep-Alive User-Agent: Apache-HttpClient/4.5.4 (Java/11.0.1 Cookie: hello=world; sushi= 🍣 Accept-Encoding: gzip,deflate
  46. 46. • Proxy • 認証有りProxy ApacheHC/Proxy var clientBuilder = HttpClientBuilder.create(); var proxy = new HttpHost(proxyHost, proxyPort, "http"); clientBuilder.setProxy(proxy); clientBuilder.setProxyAuthenticationStrategy(new ProxyAuthenticationStrategy()); var credsProvider = new BasicCredentialsProvider(); credsProvider.setCredentials( AuthScope.ANY, new UsernamePasswordCredentials(proxyUser, proxyPassword)); clientBuilder.setDefaultCredentialsProvider(credsProvider);
  47. 47. ApacheHC • Basic認証 var u = URI.create("https://httpbin.org/basic-auth/kuma/pass"); var credProvider = new BasicCredentialsProvider(); credProvider.setCredentials( new AuthScope(u.getHost(), u.getPort(), AuthScope.ANY_REALM, AuthScope.ANY_SCHEME), new UsernamePasswordCredentials("kuma", "pass")); var clientBuilder = HttpClientBuilder.create(); clientBuilder.setDefaultCredentialsProvider(credProvider); try (CloseableHttpClient client = clientBuilder.build()) { var getMethod = new HttpGet(u); try (CloseableHttpResponse resp = client.execute(getMethod)) { System.out.println(resp.getStatusLine()); } }
  48. 48. ApacheHC • Basic認証 (ANYを使う:非推奨) – 最低限、Host, Portは指定した方が良い – Realmも本当は指定した方が良いけど credProvider.setCredentials( AuthScope.ANY, new UsernamePasswordCredentials("kuma", "pass"));
  49. 49. ApacheHC/BadSSL • BadSSLスルー var sslContext = SSLContext.getInstance("SSL"); sslContext.init(null, new TrustManager[] { new X509TrustManager() { public X509Certificate[] getAcceptedIssuers() { return null; } public void checkClientTrusted(X509Certificate[] certs, String authType) { } public void checkServerTrusted(X509Certificate[] certs, String authType) { } } }, new SecureRandom()); var clientBuilder = HttpClientBuilder.create(); clientBuilder.setSSLSocketFactory(new SSLConnectionSocketFactory(sslContext)); try (CloseableHttpClient client = clientBuilder.build()) { var getMethod = new HttpGet("https://expired.badssl.com/"); try (var resp = client.execute(getMethod)) { String body = EntityUtils.toString(resp.getEntity(), StandardCharsets.UTF_8); System.out.println(body); } }
  50. 50. ApacheHC/Async • HttpAsyncClient – Basicな方法 CloseableHttpAsyncClient client = HttpAsyncClients.createDefault(); client.start(); var request1 = new HttpGet("https://httpbin.org/get"); Future<HttpResponse> future = client.execute(request1, null); // any processing var resp = future.get(); System.out.println(resp.getStatusLine()); client.close();
  51. 51. ApacheHC/Async • HttpAsyncClient – Consumer / Producer CloseableHttpAsyncClient client = HttpAsyncClients.createDefault(); client.start(); var producer = HttpAsyncMethods.create(new HttpGet("https://httpbin.org/get")); var consumer = new AsyncCharConsumer<HttpResponse>() { HttpResponse response; protected void onCharReceived(CharBuffer charBuffer, IOControl ioControl) { } protected void onResponseReceived(HttpResponse httpResponse) { this.response = httpResponse; } protected HttpResponse buildResult(HttpContext httpContext) { return this.response; } };
  52. 52. ApacheHC/Async • HttpAsyncClient – Consumer / Producer var latch = new CountDownLatch(1); client.execute(producer, consumer, new FutureCallback<>() { public void completed(HttpResponse httpResponse) { latch.countDown(); } public void failed(Exception e) { latch.countDown(); } public void cancelled() { latch.countDown(); } }); latch.await(); System.out.println(consumer.response.getStatusLine()); client.close();
  53. 53. ApacheHC/Debug • DEBUG – Loggerを設定する – Lobackの例 <logger name="org.apache.http" additivity="true"> <level value="DEBUG"/> </logger>
  54. 54. ApacheHC/不満点 • HTTP/2は まだ 対応していない • WebSocket対応もまだ
  55. 55. java.net.http
  56. 56. おさらい • java.net.http / HttpClient – Java11で正式APIになった – HTTP/2対応 – WebSocket対応 – 同期/非同期 処理対応 – Reactive Streamとして Request/Responseを処理 – Builderパターン
  57. 57. java.net.http • Clientの作り方 もしくは [Header] GET /get HTTP/1.1 Content-Length: 0 Host: httpbin.org User-Agent: Java-http-client/11.0.1 var client = HttpClient.newHttpClient(); var request = HttpRequest.newBuilder() .uri(URI.create("https://http2.pro/api/v1")) .build(); var resp = client.send(request, HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)); System.out.println(resp.statusCode()); System.out.println(resp.body()); var client = HttpClient.newBuilder().build(); [Body] {"http2":1,"protocol":"HT TP¥/2.0","push":1,"user_a gent":"Java-http- client¥/11.0.1"}
  58. 58. java.net.http • デフォルトは、 – HTTP/2を勝手にやってくれる • サイトが対応していない場合はHTTP/1.1で通信 – Redirectは勝手にしてくれない – GZip/Deflateは勝手に処理してくれない • 便利なURIBuilderは存在しない
  59. 59. java.net.http • ClientBuilder – authenticator – connectTimeout – cookieHandler – executor – followRedirects – priority – proxy – sslContext – version – sslParameters
  60. 60. java.net.http • BodyHandlers – ofString – ofByteArray – ofByteArrayConsumer – ofFile – ofFileDownload – ofInputStream – ofLines – ofPublisher – ...
  61. 61. java.net.http/redirect • Redirectの設定 var client = HttpClient.newBuilder() .followRedirects(HttpClient.Redirect.NORMAL) .build(); NEVER Redirectしない ALWAYS 常にRedirectする NORMAL 常にRedirectする(HTTPS->HTTPは除く)
  62. 62. java.net.http/Header • Request Headerの編集 var request = HttpRequest.newBuilder() .header("Accept", "*/*") .header("User-Agent", "JJUG/1.0") .uri(URI.create("https://httpbin.org/get")) .build();
  63. 63. java.net.http/POST • POSTとBody送信 var bodyString = new Gson().toJson(Map.of("hello", "world")); var client = HttpClient.newBuilder().build(); var request = HttpRequest.newBuilder() .header("Content-Type", "application/json") .POST(HttpRequest.BodyPublishers.ofString(bodyString)) .uri(URI.create("https://httpbin.org/post")) .build(); var resp = client.send( request, HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)); System.out.println(resp.statusCode()); System.out.println(resp.body());
  64. 64. java.net.http • BodyPublishers – noBody – ofString – ofByteArray – ofByteArrays – ofFile – ofInputStream – ...
  65. 65. java.net.http/method • 他のメソッド – GET(), POST(), DELETE(), PUT() – HEADやPATCHは? • methodを使う .method("HEAD", HttpRequest.BodyPublishers.noBody()) .method("PATCH", HttpRequest.BodyPublishers.ofString(bodyString))
  66. 66. java.net.http/async • 非同期: client.sendAsync var client = HttpClient.newBuilder().build(); var request = HttpRequest.newBuilder(URI.create("https://httpbin.org/get")).build(); CompletableFuture<HttpResponse<String>> future = client.sendAsync(request, HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)); // CompletableFutureなので処理を繋げる事ができる future.thenApply(HttpResponse::body) .exceptionally(e -> "Error!! :" + e.getMessage()) .thenAccept(System.out::println); //future.get();
  67. 67. java.net.http/content-encoding • BodySubscribers.mappingを使う – ※JDKの不具合で動きません https://bugs.openjdk.java.net/browse/JDK-8217264 private HttpResponse.BodySubscriber<InputStream> gzippedBodySubscriber( HttpResponse.ResponseInfo responseInfo) { // 本当はheaderのContent-Encodingを確認する return HttpResponse.BodySubscribers.mapping( HttpResponse.BodySubscribers.ofInputStream(), this::decodeGzipStream); } private InputStream decodeGzipStream(InputStream gzippedStream) { try { return new GZIPInputStream(gzippedStream); } catch (IOException ex) { throw new UncheckedIOException(ex); } } var resp = client.send(request, this::gzippedBodySubscriber);
  68. 68. java.net.http/content-encoding • Workaround-1 var client = HttpClient.newBuilder().build(); var request = HttpRequest.newBuilder() .uri(URI.create("https://httpbin.org/gzip")) .build(); var resp = client.send(request, HttpResponse.BodyHandlers.ofInputStream()); InputStream in = resp.body(); if ("gzip".equals(resp.headers().firstValue("Content-Encoding").get())) { in = new GZIPInputStream(resp.body()); } System.out.println(IOUtils.toString(in, StandardCharsets.UTF_8));
  69. 69. java.net.http/content-encoding • Workaround-2 var future = client.sendAsync(request, HttpResponse.BodyHandlers.ofInputStream()) .thenApply(resp -> { String ce = resp.headers().firstValue("Content-Encoding").get(); if ("gzip".equals(ce)) { try { return new GZIPInputStream(resp.body()); } catch (IOException e) { throw new UncheckedIOException(e); } } return resp.body(); }); var in = future.get(); System.out.println(IOUtils.toString(in, StandardCharsets.UTF_8));
  70. 70. java.net.http/content-encoding • Workaround others https://stackoverflow.com/questions/53379087/wrapping- bodysubscriberinputstream-in-gzipinputstream-leads-to-hang
  71. 71. java.net.http/custom handler • 例: json public static class JsonBodyHandler<T> implements HttpResponse.BodyHandler<T> { private Class<T> type; private JsonBodyHandler(Class<T> type) { this.type = type; } public static <T> JsonBodyHandler<T> jsonBodyHandler(Class<T> type) { return new JsonBodyHandler<>(type); } @Override public HttpResponse.BodySubscriber<T> apply( HttpResponse.ResponseInfo info) { return HttpResponse.BodySubscribers.mapping( HttpResponse.BodySubscribers.ofString(StandardCharsets.UTF_8), s -> new Gson().fromJson(s, this.type)); } } var resp = client.send(request, JsonBodyHandler.jsonBodyHandler(MyJson.class)); MyJson body = resp.body();
  72. 72. java.net.http/query-param, form-encoded • 用意されていない – 便利なライブラリが他にあるのでそれを利用して BodyにStringとして渡す – (例) Spring関連のLibrary – (例) Apache HttpClient 😨
  73. 73. • デフォルトでは無効化されている java.net.http/cookie var cookieHandler = new CookieManager(); var client = HttpClient.newBuilder().cookieHandler(cookieHandler).build(); var u = URI.create("https://httpbin.org/cookies/set/hello/world"); var request1 = HttpRequest.newBuilder(u).build(); client.send(request1, HttpResponse.BodyHandlers.ofString()); var cookie = new HttpCookie("sushi", "tenpura"); cookie.setDomain(u.getHost()); cookie.setPath("/"); cookie.setVersion(0); cookieHandler.getCookieStore().add(u, cookie); var request2 = HttpRequest.newBuilder(URI.create("https://httpbin.org/cookies")).build(); var resp2 = client.send(request2, HttpResponse.BodyHandlers.ofString()); Version0が⼤事 Version0 -> Netscape style Version1 -> RFC2965/2109
  74. 74. java.net.http/basic auth • Basic認証 – HttpUrlConnectionのサンプルと同じ – 利用クラスも同じなのでそちらを参照 var client = HttpClient.newBuilder() .authenticator(authenticator) .build();
  75. 75. java.net.http/download • Download var client = HttpClient.newBuilder() .followRedirects(HttpClient.Redirect.ALWAYS) .build(); var u = URI.create("https://github.com/AdoptOpenJDK/openjdk11- binaries/releases/download/jdk-11.0.3%2B7/OpenJDK11U- jdk_x64_linux_hotspot_11.0.3_7.tar.gz"); var request = HttpRequest.newBuilder().uri(u).build(); Path f = Paths.get("/tmp/").resolve(Path.of(u.getPath()).getFileName()); var resp = client.send(request, HttpResponse.BodyHandlers.ofFile(f)); System.out.println(resp.statusCode()); System.out.println(f.toFile().length());
  76. 76. java.net.http/proxy with authentication System.setProperty("jdk.http.auth.tunneling.disabledSchemes", ""); var proxySelector = ProxySelector.of( InetSocketAddress.createUnresolved(PROXY_HOST, PROXY_PORT)); var client = HttpClient.newBuilder() .proxy(proxySelector) .authenticator(new Authenticator() { protected PasswordAuthentication getPasswordAuthentication() { if (getRequestorType() == RequestorType.PROXY) { return new PasswordAuthentication(PROXY_USER, PROXY_PASS); } return null; } }).build(); var u = URI.create("https://httpbin.org/get"); var request1 = HttpRequest.newBuilder(u).build(); var resp = client.send(request1, HttpResponse.BodyHandlers.ofString()); https://www.oracle.com/technetwork/java/javase/8u111-relnotes-3124969.html jdk8u111より これが無いとhttpsのurlの 認証処理がskipされる Proxyの時だけ認証
  77. 77. java.net.http/badSSL • SSLContextを設定できるので、ApacheHCと同じ var sslContext = SSLContext.getInstance("TLS"); sslContext.init(null, new TrustManager[] { new X509TrustManager() { public X509Certificate[] getAcceptedIssuers() { return null; } public void checkClientTrusted(X509Certificate[] certs, String authType) { } public void checkServerTrusted(X509Certificate[] certs, String authType) { } } }, new SecureRandom()); var client = HttpClient.newBuilder().sslContext(sslContext).build(); var u = URI.create("https://expired.badssl.com/"); var request1 = HttpRequest.newBuilder(u).build(); var resp = client.send(request1, HttpResponse.BodyHandlers.ofString());
  78. 78. java.net.http/server-push https://www.nginx.com/blog/nginx-1-13-9-http2-server-push/
  79. 79. java.net.http/server-push var httpClient = HttpClient.newBuilder().version(HttpClient.Version.HTTP_2).build(); var pageRequest = HttpRequest.newBuilder(URI.create("https://http2.golang.org/serverpush")).build(); AtomicInteger cnt = new AtomicInteger(1); var futures = new CopyOnWriteArrayList<CompletableFuture<HttpResponse<String>>>(); HttpResponse.PushPromiseHandler<String> handler = (initiatingRequest, pushPromiseRequest, acceptor) -> { System.out.println("Promise request: " + pushPromiseRequest.uri()); var pushedFuture = acceptor.apply(HttpResponse.BodyHandlers.ofString()); pushedFuture = pushedFuture.thenApply(resp -> { System.out.println("[" + cnt.getAndIncrement() + "] Pushed response: " + resp.uri()); return resp; }); futures.add(pushedFuture); }; applyPushPromise( HttpRequest initiatingRequest, HttpRequest pushPromiseRequest, Function<HttpResponse.BodyHandler<String>, CompletableFuture<HttpResponse<String>>> acceptor )
  80. 80. java.net.http/server-push var future = httpClient.sendAsync( pageRequest, HttpResponse.BodyHandlers.ofString(), handler); future.thenAccept(pageResponse -> { System.out.println("Page response status code: " + pageResponse.statusCode()); }).join(); var array = futures.toArray(new CompletableFuture[0]); CompletableFuture.allOf(array).get(); Promise request: https://http2.golang.org/serverpush/static/jquery.min.js?1558076345232851844 Promise request: https://http2.golang.org/serverpush/static/godocs.js?1558076345232851844 Promise request: https://http2.golang.org/serverpush/static/playground.js?1558076345232851844 Promise request: https://http2.golang.org/serverpush/static/style.css?1558076345232851844 [1] Pushed response: https://http2.golang.org/serverpush/static/style.css?1558076345232851844 [2] Pushed response: https://http2.golang.org/serverpush/static/playground.js?1558076345232851844 [3] Pushed response: https://http2.golang.org/serverpush/static/godocs.js?1558076345232851844 Page response status code: 200 [4] Pushed response: https://http2.golang.org/serverpush/static/jquery.min.js?1558076345232851844
  81. 81. java.net.http/web-socket var u = URI.create("wss://echo.websocket.org"); var client = HttpClient.newBuilder().build(); var listener = new WebSocket.Listener() { public CompletionStage<?> onText(WebSocket webSocket, CharSequence data, boolean last) { System.out.println("receive:" + data + ", last=" + last); return WebSocket.Listener.super.onText(webSocket, data, last); } }; WebSocket wsock = client.newWebSocketBuilder().buildAsync(u, listener).join(); wsock.sendText("hello world 1", false); wsock.sendText("hello world 2", true); TimeUnit.SECONDS.sleep(2); wsock.sendClose(WebSocket.NORMAL_CLOSURE, "BYE");
  82. 82. java.net.http/debug [class] jdk.internal.net.http.common.Utils jdk.internal.net.http.common.DebugLogger jdk.internal.net.http.common.Log jdk.internal.net.http.common.Logger • システムプロパティの設定 jdk.internal.httpclient.debug=true jdk.internal.httpclient.websocket.debug=true jdk.internal.httpclient.hpack.debug=true
  83. 83. • もしくは • もしくは – LoggerName java.net.http/debug -Djava.net.HttpClient.log= errors,requests,headers,frames[:control:data:window:all..],co ntent,ssl,trace,channel jdk.httpclient.HttpClient
  84. 84. java.net.http • reactive streamの話をしていない?
  85. 85. java.net.http • reactive streamの話をしていない? – 今まで使っていたPublisher, SubScriberが そうなのです。 public interface BodySubscriber<T> extends Flow.Subscriber<List<ByteBuffer>> { CompletionStage<T> getBody(); void onSubscribe(Subscription subscription); void onNext(List<ByteBuffer> item); void onError(Throwable throwable); void onComplete(); } public interface BodyPublisher extends Flow.Publisher<ByteBuffer> { long contentLength(); void subscribe(Subscriber<? super ByteBuffer> subscriber); }
  86. 86. java.net.http • もっといろいろな書き方が出来ます!! – Javadocと公式サンプルを見て遊びましょう 続く(かも)

×