This document provides an overview and agenda for an AWS webinar on AWS Glue. It introduces AWS Glue as a fully managed and serverless ETL service that can manage metadata for various data sources. The webinar will cover the background of AWS Glue, its key features including being serverless and enabling secure development in notebooks, use cases, pricing, and a conclusion. It also provides details on the components and functions of AWS Glue like the data catalog, orchestration, and serverless engines.
This document provides information about an AWS webinar on AWS Step Functions hosted by Yuta Imamura from Amazon Web Services Japan. The agenda includes an overview of Step Functions, state machines, data input and output, describing states, checking execution status, and additional details. Step Functions allows orchestrating distributed applications and microservices using state machines defined in Amazon States Language (ASL). States can pass data and parameters between each other to synchronize processes.
20190521 AWS Black Belt Online Seminar Amazon Simple Email Service (Amazon SES)Amazon Web Services Japan
This document summarizes an AWS webinar on Amazon Simple Email Service (Amazon SES). It discusses how Amazon SES can be used to send emails using SMTP or APIs. It also covers topics like delivery optimization, reputation management, and monitoring email sending activity. Configuration options like dedicated IPs and suppression lists are presented as ways to improve deliverability.
This document provides an overview and agenda for an AWS webinar on AWS Glue. It introduces AWS Glue as a fully managed and serverless ETL service that can manage metadata for various data sources. The webinar will cover the background of AWS Glue, its key features including being serverless and enabling secure development in notebooks, use cases, pricing, and a conclusion. It also provides details on the components and functions of AWS Glue like the data catalog, orchestration, and serverless engines.
This document provides information about an AWS webinar on AWS Step Functions hosted by Yuta Imamura from Amazon Web Services Japan. The agenda includes an overview of Step Functions, state machines, data input and output, describing states, checking execution status, and additional details. Step Functions allows orchestrating distributed applications and microservices using state machines defined in Amazon States Language (ASL). States can pass data and parameters between each other to synchronize processes.
20190521 AWS Black Belt Online Seminar Amazon Simple Email Service (Amazon SES)Amazon Web Services Japan
This document summarizes an AWS webinar on Amazon Simple Email Service (Amazon SES). It discusses how Amazon SES can be used to send emails using SMTP or APIs. It also covers topics like delivery optimization, reputation management, and monitoring email sending activity. Configuration options like dedicated IPs and suppression lists are presented as ways to improve deliverability.
The document appears to be a presentation on Amazon EMR and related AWS services. It discusses using EMR for big data and analytics workloads, how to set up EMR clusters on AWS, encryption options for data at rest and in transit, and integration of EMR with other AWS services like S3 and Spark. The presentation contains many bullet points and diagrams but provides little surrounding context or narrative.
The document discusses Amazon Web Services (AWS) Batch and how it can help customers run batch computing workloads on AWS. It notes that AWS Batch automatically provisions the optimal quantity and type of compute resources (e.g., EC2 instances) required to run jobs efficiently. It also allows customers to integrate their own scheduling and application code with AWS Batch through simple API calls or SDKs.
This document provides an overview and agenda for a webinar on Amazon Elasticsearch Service (Amazon ES). It introduces Elasticsearch and Amazon ES, including the benefits of using the fully-managed service such as cost efficiency, high availability, security, and ease of deployment. The webinar will cover topics like log analysis, search capabilities, operations management, security, pricing and limits.
20200422 AWS Black Belt Online Seminar Amazon Elastic Container Service (Amaz...Amazon Web Services Japan
This document provides an overview and agenda for an AWS webinar on Amazon Elastic Container Service (ECS). The webinar will cover the basics of ECS, its main components, container execution environments, key ECS features, task definitions, and container execution. It includes introductions to the presenter and notices about pricing and content details. The target audience is those considering using containers in production environments on AWS.
This document discusses DNS and Amazon Route 53. It begins with an overview of DNS records like NS, A, AAAA, CNAME, PTR and MX records. It then covers DNS concepts such as the domain name system, domain name registration and resolution. The document also discusses how Route 53 can be used to configure DNS settings across public and private hosted zones for routing traffic to AWS resources.
The document appears to be a presentation on Amazon EMR and related AWS services. It discusses using EMR for big data and analytics workloads, how to set up EMR clusters on AWS, encryption options for data at rest and in transit, and integration of EMR with other AWS services like S3 and Spark. The presentation contains many bullet points and diagrams but provides little surrounding context or narrative.
The document discusses Amazon Web Services (AWS) Batch and how it can help customers run batch computing workloads on AWS. It notes that AWS Batch automatically provisions the optimal quantity and type of compute resources (e.g., EC2 instances) required to run jobs efficiently. It also allows customers to integrate their own scheduling and application code with AWS Batch through simple API calls or SDKs.
This document provides an overview and agenda for a webinar on Amazon Elasticsearch Service (Amazon ES). It introduces Elasticsearch and Amazon ES, including the benefits of using the fully-managed service such as cost efficiency, high availability, security, and ease of deployment. The webinar will cover topics like log analysis, search capabilities, operations management, security, pricing and limits.
20200422 AWS Black Belt Online Seminar Amazon Elastic Container Service (Amaz...Amazon Web Services Japan
This document provides an overview and agenda for an AWS webinar on Amazon Elastic Container Service (ECS). The webinar will cover the basics of ECS, its main components, container execution environments, key ECS features, task definitions, and container execution. It includes introductions to the presenter and notices about pricing and content details. The target audience is those considering using containers in production environments on AWS.
This document discusses DNS and Amazon Route 53. It begins with an overview of DNS records like NS, A, AAAA, CNAME, PTR and MX records. It then covers DNS concepts such as the domain name system, domain name registration and resolution. The document also discusses how Route 53 can be used to configure DNS settings across public and private hosted zones for routing traffic to AWS resources.
AWS Black Belt Online Seminarの最新コンテンツ: https://aws.amazon.com/jp/aws-jp-introduction/#new
過去に開催されたオンラインセミナーのコンテンツ一覧: https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/
AWS Black Belt Online Seminarの最新コンテンツ: https://aws.amazon.com/jp/aws-jp-introduction/#new
過去に開催されたオンラインセミナーのコンテンツ一覧: https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/
AWS Black Belt Online Seminarの最新コンテンツ: https://aws.amazon.com/jp/aws-jp-introduction/#new
過去に開催されたオンラインセミナーのコンテンツ一覧: https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/
AWS Japan YouTube 公式チャンネルでライブ配信された 2022年4月26日の AWS Developer Live Show 「Infrastructure as Code 談議 2022」 の資料となります。 当日の配信はこちら からご確認いただけます。
https://youtu.be/ed35fEbpyIE
AWS Black Belt Online Seminarの最新コンテンツ: https://aws.amazon.com/jp/aws-jp-introduction/#new
過去に開催されたオンラインセミナーのコンテンツ一覧: https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/
202204 AWS Black Belt Online Seminar Amazon Connect Salesforce連携(第1回 CTI Adap...Amazon Web Services Japan
AWS Black Belt Online Seminarの最新コンテンツ: https://aws.amazon.com/jp/aws-jp-introduction/#new
過去に開催されたオンラインセミナーのコンテンツ一覧: https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/
AWS Black Belt Online Seminarの最新コンテンツ: https://aws.amazon.com/jp/aws-jp-introduction/#new
過去に開催されたオンラインセミナーのコンテンツ一覧: https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/
* AWS Black Belt Online Seminarの最新コンテンツ: https://aws.amazon.com/jp/aws-jp-introduction/#new
* 過去に開催されたオンラインセミナーのコンテンツ一覧: https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/
企業間の連携においてもSaaS活用シフトが進む一方で、インターネット経由というイメージからセキュリティーに不安を感じて踏みとどまるユーザーは多くいます。こうした懸念を払しょくするAWS PrivateLinkを活用した企業間のプライベート接続や閉域網との構成例、SaaS事業者様からなるPrivateLinkパートナーコミュニティ形成の取り組みをご紹介します。
2021年12月9日に開催された「SaaS on AWS Day 2022」での講演内容です。
パッケージソフトウェアをお持ちのお客様が新たにSaaS版のアプリケーションを検討したいというニーズが増えています。一方で"SaaS版を作っても成功するかわからない"、"WEBアプリケーションを作る技術力や知見がない"といった不安からSaaS化における課題があることも事実です。本セッションでは、小さく早くSaaSビジネスを始めたいお客様に向けて、Amazon AppStream2.0を用いた既存アプリケーションのSaaS化手法をご紹介します。
2021年12月9日に開催された「SaaS on AWS Day 2021」での講演内容です。
AWS Black Belt Online Seminarの最新コンテンツ: https://aws.amazon.com/jp/aws-jp-introduction/#new
過去に開催されたオンラインセミナーのコンテンツ一覧: https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/
AWS Black Belt Online Seminarの最新コンテンツ: https://aws.amazon.com/jp/aws-jp-introduction/#new
過去に開催されたオンラインセミナーのコンテンツ一覧: https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/
AWS Black Belt Online Seminarの最新コンテンツ: https://aws.amazon.com/jp/aws-jp-introduction/#new
過去に開催されたオンラインセミナーのコンテンツ一覧: https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/
202201 AWS Black Belt Online Seminar Apache Spark Performnace Tuning for AWS ...Amazon Web Services Japan
AWS Black Belt Online Seminarの最新コンテンツ: https://aws.amazon.com/jp/aws-jp-introduction/#new
過去に開催されたオンラインセミナーのコンテンツ一覧: https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/
8. 8 @awscloud_jp #awsblackbelt
Redshiftの構成②
SELECT *
FROM lineitem;
SQLをコンパイル、
コードを生成し、コン
ピュートノードへ配信
CPU CPU CPU CPU CPU CPU
Leaderノード
Computeノード
スライス=
メモリとディスクを
ノード内で分割した論
理的な処理単位
コンピュートノードの追
加でパフォーマンス向上
(スケールアウト)
28. 28 @awscloud_jp #awsblackbelt
Interleaved Sort Keyのデータ配置イメージ
DeptId LocId
1 A
1 B
1 C
1 D
2 A
2 B
2 C
2 D
DeptId LocId
3 A
3 B
3 C
3 D
4 A
4 B
4 C
4 D
Compound Sort Key Interleaved Sort Key
DeptId LocId
1 A
1 B
2 A
2 B
1 C
1 C
2 D
2 D
DeptId LocId
3 A
3 B
4 A
4 B
3 C
3 D
4 C
4 D
DeptId = 1 -> 1 block
LocId = C -> 4 block
DeptId = 1 -> 2 block
LocId = C -> 2 block
DeptId=1 and LocId=C-> 1 block
32. 32 @awscloud_jp #awsblackbelt
EVEN vs. DISTKEY(1)
• EVEN • DISTKEY=p_partkey
select trim(name) tablename, slice,
sum(rows)
from stv_tbl_perm where name='part'
group by name, slice
order by slice;
tablename | slice | sum
-----------+-------+---------
part | 0 | 1600000
part | 1 | 1600000
…
part | 126 | 1600000
part | 127 | 1600000
tablename | slice | sum
-----------+-------+---------
part | 0 | 1596925
part | 1 | 1597634
…
part | 126 | 1610452
part | 127 | 1596154
各スライスに均等に分散 キーのカーディナリティに依存
33. 33 @awscloud_jp #awsblackbelt
EVEN vs. DISTKEY(2)
• DISTKEY = p_brand
tablename | slice | sum
-----------+-------+---------
part | 0 | 0
part | 1 | 0
part | 2 | 0
part | 3 | 0
part | 4 | 8193350
…
part | 118 | 8193342
part | 119 | 0
part | 120 | 16384823
part | 121 | 8191943
カーディナリティの低い
カラムでは、データの極端な
偏りが生じる場合がある
= クエリー処理効率の低下
34. 34 @awscloud_jp #awsblackbelt
ALL
• 全レコードが各ノードの特定スライスに集約
tablename | slice | sum
-----------+-------+---------
part | 0 |204800000
part | 1 | 0
part | 2 | 0
part | 3 | 0
part | 4 | 0
…
part | 96 |204800000
part | 97 | 0
part | 98 | 0
…
…
各ノードの先頭スライスに
全レコードが格納される。
35. 35 @awscloud_jp #awsblackbelt
コロケーション(1)
• 関連するレコードのコロケーション
– ジョイン対象となるレコードを同一ノードに集める
• コロケーションの方法
1. ジョインに使用するカラムをDISTKEYとして作成 または
2. 分散方式 ALLでテーブルを作成(マスター・テーブルなど)
select sum(l_extendedprice* (1 - l_discount)) as revenue
from lineitem, part
Where (p_partkey = l_partkey …
1. それぞれをDISTKEYとして作成
または
2. テーブルをALLで作成
36. 36 @awscloud_jp #awsblackbelt
コロケーション(2):DISTKEY
6200995 | almond pale linen
| Manufacturer#3| Brand#32
part
lineitem
5024338535 | 6200995 | 0.01
|0.08 | A | F
|1992-01-02 | 1992-02-14
2201039 | almond pale linen
| Manufacturer#1| Brand#11
part
lineitem
121932093 | 2201039 | 0.05
|0.43 | D | E
|1994-07-11 | 1994-08-23
59. 59 @awscloud_jp #awsblackbelt
WLMパラメータとパラメータの動的変更
• 新しくWLMのパラメータにdynamicとstaticの区別が用意され、
dynamicはRedshiftを再起動せずにパラメータ変更が可能に
• dynamic parameter
– Concurrency(並列実行数),
– Percent of memory to use (メモリ使用量)
• static parameter
– User groups
– User group wildcard
– Query groups
– Query group wildcard
– Timeout
New!!
62. 62 @awscloud_jp #awsblackbelt
UDF定義の例
• 例)引数aとbを比較して大きい方を返すUDF
CREATE FUNCTION f_greater (a float, b float)
RETURNS float STABLE
AS $$
if a > b: return a
return b
$$ LANGUAGE plpythonu;
SELECT f_greater (c1, c2) FROM t1