Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

20181205 AWS Black Belt Online Seminar Amazon Athena (20190510update)

825 views

Published on

AWS公式オンラインセミナー: https://amzn.to/JPWebinar
過去資料: https://amzn.to/JPArchive

Published in: Technology
  • Be the first to comment

20181205 AWS Black Belt Online Seminar Amazon Athena (20190510update)

  1. 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Makoto Shimura, Solutions Architect 2018/12/05 Amazon Athena [AWS Black Belt Online Seminar]
  2. 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive • • • ⎼ Amazon Athena ⎼ AWS Glue ⎼ Amazon SageMaker
  3. 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive AWS Black Belt Online Seminar • • ① 吹き出しをクリック ② 質問を入力 ③ Sendをクリック Twitter #awsblackbelt
  4. 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive • 2018 12 05 AWS (http://aws.amazon.com) • AWS AWS • • AWS does not offer binding price quotes. AWS pricing is publicly available and is subject to change in accordance with the AWS Customer Agreement available at http://aws.amazon.com/agreement/. Any pricing information included in this document is provided only as an estimate of usage charges for AWS services based on certain information that you have provided. Monthly charges will be based on your actual use of AWS services, and may vary from the estimates provided.
  5. 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Agenda • Amazon Athena • • Amazon Athena • Update & Tips
  6. 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena Amazon S3 SQL
  7. 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena サーバレスでインフラ管理の必要なし 大規模データに対しても高速なクエリ 事前のデータロードなしに Amazon S3 に直接クエリ スキャンしたデータに対しての従量課金 JDBC / ODBC / API 経由で BI ツールやシステムと連携
  8. 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena DW BI Web ETL
  9. 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Presto: • Amazon Athena • • • https://prestodb.io/overview.html 参考: Presto のアーキテクチャ
  10. 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive AWS Glue Data Catalog • Amazon Athena AWS Glue Data Catalog • DB / Table / View / Partition • Data Catalog Apache Hive Metastore • AWS Glue Amazon Athena
  11. 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive AWS Glue Data Catalog • AWS Glue Amazon Athena AWS Glue Data Catalog • Amazon Athena https://docs.aws.amazon.com/ja_jp/athen sa/latest/ug/glue-faq.html
  12. 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena • • AWS Glue S3 Crawler • schema-on-read CREATE EXTERNAL TABLE IF NOT EXISTS action_log ( user_id string, action_category string, action_detail string year int, month int, ) PARTITIONED BY (year int, month int) STORED AS PARQUET LOCATION 's3://athena-examples/action-log/’ TBLPROPERTIES ('PARQUET.COMPRESS'='SNAPPY');
  13. 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena tinyint, smallint, int, bigint, boolean, float, double, string, binary, timestamp, decimal, date, varchar, char array<data_type> map map<primitive_type, data_type> struct * struct<col_name: data_type> union UNIONTYPE<data_type, data_type…> * https://aws.amazon.com/jp/blogs/big-data/create-tables-in-amazon-athena-from-nested-json-and-mappings-using-jsonserde/
  14. 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena SerDe CSV LazySimpleSerDe OpenCSVSerDe TSV LazuSimpleSerDe ‘¥t’ LazuSimpleSerDe JSON HiveJSONSerDe OpenXJsonSerDe Apache Avro AvroSerDe ORC ORCSerDe Apache Parquet ParquetSerDe Logstash Grok SerDe Apache RegexSerDe CloudTrail CloudTrailSerDe OpenXJSONSerDe https://docs.aws.amazon.com/ja_jp/athena/latest/ug/supported-format.html
  15. 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena SNAPPY Parquet ZLIB ORC GZIP 1GB LZO
  16. 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena • Presto ANSI SQL • WITH Window JOIN • Presto 0.172 [ WITH with_query [, ...] ] SELECT [ ALL | DISTINCT ] select_expression [, ...] [ FROM from_item [, ...] ] [ WHERE condition ] [ GROUP BY [ ALL | DISTINCT ] grouping_element [, ...] ] [ HAVING condition ] [ UNION [ ALL | DISTINCT ] union_query ] [ ORDER BY expression [ ASC | DESC ] [ NULLS FIRST | NULLS LAST] [, ...] ] [ LIMIT [ count | ALL ] ]
  17. 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena DDL / • EXTERNAL TABLE S3 Athena • 20 • DDL / 20 • VIEW CTAS • • • UDF / UDAF • • DDL http://docs.aws.amazon.com/athena/latest/ug/language-reference.html#unsupported-ddl
  18. 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena API • API 2 • API JDBC • StartQueryExecution QueryExecutionId • GetQueryExecution • State SUCCEEDED GetQueryResults Named Query API BatchGetNamedQuery CreatenamedQuery DeleteNamedQuery GetNamedQuery ListNamedQueries Query Execution API BatchGetQueryExecution GetQueryExecution GetQueryResutls ListQueryExecutions StartQueryExecutions StopQueryExecutions
  19. 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive
  20. 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Athena • OLTP (Online Transactional Processing) OLAP (Online Analytical Processing) • • ETL • & • • • • •
  21. 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive OLAP • WHERE GROUP BY • WHERE SELECT • • • SELECT col1 , col2 , COUNT(col3) , SUM(col3) FROM table1 INNER JOIN table2 ON table1.id = table2.id WHERE table1.id = table2.id AND col4 = 1 AND col5 = “good” GROUP BY col1 , col2
  22. 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive • S3 CREATE TABLE • WHERE • 1 1,000,000 AWS Glue Data Catalog Amazon Athena 20,000 CREATE EXTERNAL TABLE IF NOT EXISTS action_log ( user_id string, action_category string, year int, month int, day int ) PARTITIONED BY (year int, month int, day int) STORED AS PARQUET LOCATION 's3://athena-examples/action-log/’ TBLPROPERTIES ('PARQUET.COMPRESS'='SNAPPY'); s3://athena-examples/action-log/year=2017/month=03/day=01/data01.gz
  23. 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive • WHERE • • • “year/month/day” SELECT month , action_category , action_detail , COUNT(user_id) FROM action_log WHERE year = 2016 AND month >= 4 AND month < 7 GROUP BY month , action_category , action_detail 以下の Amazon S3 パスだけが読み込まれる s3://athena-examples/action-log/year=2016/month=04/day=01/ s3://athena-examples/action-log/year=2016/month=04/day=02/ s3://athena-examples/action-log/year=2016/month=04/day=03/ ... s3://athena-examples/action-log/year=2016/month=07/day=31/
  24. 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive (Hive ) col1=val1/col2=val2/ • CREATE TABLE MSCK REPAIR TABLE OK • MSCK REPAIR TABLE 1 OK • val1/val2/ • • MSCK REPAIR TABLE ALTER TABLE ADD PARTITION • ALTER TABLE ADD PARTITION
  25. 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive • • • OLAP • ORC, Parquet • • 1 • OLTP • TEXTFILE(CSV, TSV) ORCのデータ構造 https://orc.apache.org/docs/spec-intro.html
  26. 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 列指向行指向 I/O の効率があがる • 圧縮と同時に使うことで I/O 効率がさらに向上 • カラムごとに分けられてデータが並んでいる • 同じカラムは,似たような中身のデータが続く ため,圧縮効率がよくなる 1 2 3 4 5 61 2 3 4 5 6 a 列指向行指向 OLAP 系の分析クエリを効率的に実行できる • たいていの分析クエリは,一度のクエリで 一部のカラムしか使用しない • 単純な統計データなら,メタデータで完結 する
  27. 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive & • CREATE • & • SELECT month , action_category , COUNT(action_category) FROM action_log WHERE year = 2016 AND month >= 4 AND month < 7 GROUP BY month , action_category CREATE EXTERNAL TABLE IF NOT EXISTS action_log ( user_id string, action_category string, action_detail string year int, month int, day int ) PARTITIONED BY (year int, month int, day int) STORED AS PARQUET LOCATION 's3://athena-examples/action-log/’ TBLPROPERTIES ('PARQUET.COMPRESS'='SNAPPY');
  28. 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive CREATE EXTERNAL TABLE IF NOT EXISTS elb_logs_pq ( request_timestamp string, elb_name string, request_ip string, request_port int, … user_agent string, ssl_cipher string, ssl_protocol string ) PARTITIONED BY (year int, month int, day int) STORED AS PARQUET LOCATION 's3://athena-examples/elb/parquet/’ TBLPROPERTIES("parquet.compress"="SNAPPY"); CREATE EXTERNAL TABLE IF NOT EXISTS elb_logs_raw ( request_timestamp string, elb_name string, request_ip string, request_port int, backend_ip string, … user_agent string, ssl_cipher string, ssl_protocol string ) PARTITIONED BY(year string, month string, day string) ROW FORMAT SERDE ’org.apache.hadoop.hive.serde2.RegexSerDe’ WITH SERDEPROPERTIES ( 'serialization.format' = '1', 'input.regex' = '([^ ]*) ([^ ]*) ([^ ]*):… ([A-Za-z0-9.-]*)$’ ) LOCATION 's3://athena-examples/elb/raw/'; Parquet & Snappy Regex & Raw file
  29. 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive https://aws.amazon.com/jp/blogs/news/analyzing-data-in-s3-using-amazon-athena/ SELECT elb_name, uptime, downtime, cast(downtime as double)/cast(uptime as double) uptime_downtime_ratio FROM (SELECT elb_name, sum(case elb_response_code WHEN '200' THEN 1 ELSE 0 end) AS uptime, sum(case elb_response_code WHEN '404' THEN 1 ELSE 0 end) AS downtime FROM elb_logs_pq GROUP BY elb_name ) S3 Regex & Raw file 1TB 236s 1.15TB $5.75 Parquet & Snappy 130GB 6.78s 2.51GB $0.013 / 87% 34 99% 99.7%
  30. 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena Top 10 (1) 1. WHERE 2. IO 3. 128 MB 4. Parquet / ORC 5. ORDER BY LIMIT ORDER BY https://aws.amazon.com/jp/blogs/news/top-10-performance-tuning-tips-for-amazon-athena/
  31. 31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena Top 10 (2) 6. JOIN 7. GROUP BY 8. LIKE LIKE RegEx 9. COUNT DISTINCT APPROX_DISTINCT() 10. * SELECT https://aws.amazon.com/jp/blogs/news/top-10-performance-tuning-tips-for-amazon-athena/
  32. 32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena
  33. 33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive CASE 1: • Amazon S3 CSV • CTAS Parquet • VIEW • • Amazon QuickSight BI S3 BI
  34. 34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive CTAS (create table as select) • / • CREATE TABLE new_table WITH ( format = ‘PARQUET’, external_location = ‘s3://my_athena_results/new_table/’ partitioned_by = ARRAY[‘year’, ‘month’, ‘day’], bucketed_by = ARRAY[‘user_id’, ‘category’], bucket_count = 3, ) AS SELECT * FROM old_table;
  35. 35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive CTAS • ARRAY[‘key1’, ‘key2’, ‘key3’] s3://MYBUCKET/key1=xxx/key2=yyy/key3=zzz • CTAS 100 • • • WHERE • 128-512MB •
  36. 36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive VIEW • VIEW • VIEW VIEW • • AWS Glue Data Catalog CREATE VIEW my_view AS SELECT col1, col2 / 100 AS col2_percent FROM table1 INNER JOIN table2 ON table1.id = table2.id;
  37. 37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive ESRI Java Geometry Library • point line multiline • distance equals overlaps • 2 • ST_CONTAINS, ST_POINT SELECT counties.name, COUNT(*) cnt FROM counties CROSS JOIN earthquakes WHERE ST_CONTAINS ( counties.boundaryshape, ST_POINT(earthquakes.longitude, earthquakes.latitude)) GROUP BY counties.name ORDER BY cnt DESC https://docs.aws.amazon.com/ja_jp/athena/latest/ug/geospatial-query-what-is.html
  38. 38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon QuickSight Amazon Athena • Amazon QuickSight JDBC/ODBC GUI Amazon Athena • Amazon Athena • Amazon Athena SPICE Amazon Athena https://docs.aws.amazon.com/ja_jp/quicksight/latest/user/create-a-data-set-athena.html
  39. 39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive JDBC / ODBC BI • BI SQL • IAM API • IAM athena:GetQueryResultsStream • JDBC 2.0.5 • ODBC: 1.0.3 https://docs.aws.amazon.com/ja_jp/athena/latest/ug/athena-bi-tools-jdbc-odbc.html URL: jdbc:awsathena://athena.${REGION}.amazonaws.com:443 Username: $AWS_ACCESS_KEY_ID Password: $AWS_SECRET_ACCESS_KEY
  40. 40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive JDBC / ODBC MS Active Directory • JDBC / ODBC Active Directory IAM Athena https://docs.aws.amazon.com/athena/latest/ug/access-federation-saml.html URL: jdbc:awsathena://athena.${REGION}.amazonaws.com:443 AwsCredentialProviderClass: com.simba.athena.iamsupport.plugin.AdfsCredentialsProvider Idp_host: example.adfs.server Idp_prot: 223 UID: $YOUR_AD__UID PWD: $YOUR_AD_PASSWORD Preferred_role: arn:aws:iam::$YOUR_AWS_ACCOUNT:role/$IAMUSER_NAME
  41. 41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive CASE 2: ETL Amazon Athena • JSON • CTAS Parquet • AWS Glue Data Catalog S3 C A B A B C
  42. 42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive CTAS Amazon Athena INSERT INTO INSERT INTO 1. CREATE EXTERNAL TABLE s3://MY_BUCKET/service_a/ table_service_a 2. CTAS tmp_table_service_a • external_location = s3://MY_BUCKET/service_a/year=2018/month=12/day=01 • Bucketed_by = ARRAY[‘high_cardinality_col’] • bucket_count = XX 3. table_service_a ALTER TABLE ADD PARTITION 4. tmp_table_service_a DELETE TABLE 5. 2-4
  43. 43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive AWS Glue Data Catalog resource-based policy • AWS Glue Data Catalog IAM • Amazon Athena • AWS Glue Data Catalog Amazon S3 Amazon S3 • S3 Amazon Athena Amazon S3 API
  44. 44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Case 3: • Amazon Kinesis Data Firehose • Amazon S3 Parquet • •
  45. 45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Kinesis Data Firehose Parquet • S3 Amazon Kinesis Data Firehose • 2018/5/10 Parquet / OCR • AWS Lambda • Amazon Athena
  46. 46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive ALTER TABLE ADD PARTITION • Amazon Athena • • 10 ALTER TABLE kdf_log ADD PARTITION (year=2018, month=12, day=5) LOCATION ‘s3://MY_KDF_BUCKET/2018/12/05/’; ALTER TABLE kdf_log ADD PARTITION (year=2018, month=12, day=5) LOCATION ‘s3://MY_KDF_BUCKET/2018/12/06/’; ALTER TABLE kdf_log ADD PARTITION (year=2018, month=12, day=5) LOCATION ‘s3://MY_KDF_BUCKET/2018/12/07/’; ... ALTER TABLE kdf_log ADD PARTITION (year=2018, month=12, day=5) LOCATION ‘s3://MY_KDF_BUCKET/2025/12/31/’;
  47. 47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Update & Tips
  48. 48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Athena Workgroups • • • https://www.slideshare.net/AmazonWebServices/amazon-athena-whats-new-and-how-sendgrid-innovates-ant324-aws-reinvent-2018
  49. 49. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive • Athena • Athena 5 https://www.slideshare.net/AmazonWebServices/amazon-athena-whats-new-and-how-sendgrid-innovates-ant324-aws-reinvent-2018
  50. 50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive • • Amazon S3 • Amazon S3 • 3 • SSE-S3 • SSE-KMS • CSE-KMS • https://docs.aws.amazon.com/ja_jp/athena/latest/ug/encryption.html
  51. 51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive AWS Amazon Athena S3 AWS • AWS CloudTrail • Amazon CloudFront • Elastic Load Balancing (ALB/CLB) • Amazon VPC • AWS Cost and Usage Reports • AWS Systems Manager https://docs.aws.amazon.com/ja_jp/athena/latest/ug/querying-AWS-service-logs.html https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/athena.html https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-inventory-query.html
  52. 52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena 1 • : 20 (DDL 20 ) • : 30 • 1 API API 名 バースト時の値 BatchGetNamedQuery,BatchGetQueryExecution ListNamedQueries, ListQueryExecutions 5 最大 10 CreateNamedQuery, DeleteNamedQuery, GetNamedQuery StartQueryExecution, StopQueryExecution 5 最大 20 GetQueryExecution, GetQueryResults 25 最大 50 https://docs.aws.amazon.com/athena/latest/ug/service-limits.html
  53. 53. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Glue 1 • : 10,000 • : 100,000 • : 1,000,000 • : 1,000,000 • : 10,000,000
  54. 54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive • S3 1TB 5$ • 10MB 10MB • MB • S3 • DDL • Amazon S3 S3 • Amazon S3 API S3 • AWS Glue Data Catalog
  55. 55. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Amazon Athena • • • • • • • • • •
  56. 56. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive
  57. 57. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive • Amazon Athena Amazon S3 SQL • • AWS Glue Amazon Kinesis Data Firehose
  58. 58. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive Q&A AWS Japan Blog https://aws.amazon.com/jp/blogs/news/
  59. 59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive AWS Black Belt Online Seminar 12月開催予定 12/11 (火) 12:00-13:00 AWS Well-Architected ( ) 12/18 (火) 12:00-13:00 Amazon Sumerian 12/19 (水) 18:00-19:00 AWS Certificate Manager 12/25 (火) 12:00-13:00 Amazon DynamoDB Advanced Design Pattern 申し込み https://amzn.to/JPWebinar
  60. 60. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Webinar https://amzn.to/JPWebinar | https://amzn.to/JPArchive

×