More Related Content
Similar to DB-Freedom資料建構營-應用案例分享-Matt Wu (20)
More from Amazon Web Services (20)
DB-Freedom資料建構營-應用案例分享-Matt Wu
- 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
“The world’s most valuable
resource is
no longer oil, but data.”*
*Copyright: The Economist, 2017, David Parkins
資料是每一個公司或是組織新
的戰略資源
- 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
趨勢及我們的觀察
1. 資料爆炸性的增長
2. 雲端服務將幫助客戶使用更多樣的架構
3. 不同場景應該利用不同型態資料庫
4. 所有的資料都有價值 (90%的資料因為成本因素而丟棄)
- 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
雲端已經成為一個常態
在雲端打造全新應用 把現有的架構搬上雲端
- 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
在雲端打造全新應用
- 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
新的趨勢造就新的應用場景
Users: 1million+
Data volume: TB–PB–EB
Locality: Global
Performance: Milliseconds–microseconds
Request Rate: Millions
Access: Web, Mobile, IoT, devices
Scale: Up-down, Out-in
Economics: Pay for what you use
Developer access: Instant API access
Social mediaRide hailing Media streaming Dating
- 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
挑選最合適的工具與架構來解決問題
利用代管服務來降低維運的成本與增加快速擴展的能力
搭配不同型態的資料庫來滿足應用需求
Developers想要專注於開發, 快速打造出應用
- 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Expedia
實時資料串流處理: ElastiCache
交易資料庫: RDS
資料倉儲: Redshift
- 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
全球25M+用戶
>1M 車次/天,
8x traffic in peak hours
DynamoDB儲存所有GPS
路徑資料
- 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Entire analytics platform running on AWS
Amazon S3 leveraged as a data lake
All telemetry data is collected with Amazon Kinesis
Real-time analytics done through Spark on Amazon EMR,
DynamoDB to create scoreboards and real-time queries
Use Amazon EMR for large batch data processing
Game designers use data to inform their decisions
Game
clients
Game
servers
Launcher
Game
services
N E A R R E A L T I M E P I P E L I N E
N E A R R E A L T I M E P I P E L I N E
Grafana
Scoreboards API
Limited raw data
(real time ad-hoc SQL)
User ETL
(metric definition)
Spark on EMR DynamoDB
NEAR REAL-TIME PIPELINES
BATCH PIPELINES
ETL using
EMR
Tableau/BI
Ad-hoc SQLS3
(Data lake)
Kinesis
APIs
Databases
S3
Other
sources
- 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
把現有的架構搬上雲端
- 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
硬體與軟體安裝與維護
資料庫configuration, patching, and backups
Cluster管理與資料複寫來達成高可用性
難以預估系統成長與規劃硬體需求
客戶告訴我們
管理地端的資料庫很複雜又很花時間
- 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon RDS
全代管的雲端資料庫
易於管理
Easily deploy and maintain
hardware, OS and DB
software; built-in monitoring
高可用性
Automatic Multi-AZ data
replication; automated
backup, snapshots, and
failover
高性能, 容易擴充
Scale compute
and storage with a few
clicks; minimal downtime for
your application
安全與合規
Data encryption at rest and
in transit; industry
compliance and assurance
programs
- 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
數以萬計的客戶正在使用Amazon RDS
- 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
另外, 客戶告訴我們他們面臨的狀況
AUDIT
Very expensive Proprietary Lock-in Punitive
licensing
You’ve got mail
- 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
+
商用等級資料庫的性能與可靠度?
- 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Aurora
兼容於MySQL and PostgreSQL的雲端原生資料庫
商用資料庫等級的性能與可靠度, 只要十分之一的價格
高性能, 易於擴充
5x throughput of standard
MySQL and 3x of standard
PostgreSQL; scale-out up
to15 read replicas
高可用性, 高可靠度
Fault-tolerant, self-healing
storage; six copies of
data across three AZs;
continuous backup to S3
高安全性
Network isolation,
encryption at
rest/transit
全代管
Managed by RDS: no
server provisioning,
software patching, setup,
configuration, or backups
- 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Aurora是AWS有史以來成長最快速的服務
- 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
雲端搬遷工具與資源
- 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
M I G R A T I N G
D A T A B A S E S
T O A W S
Migrate between on-premises and AWS
Migrate between databases
Automated schema conversion
Data replication for zero
downtime migration
- 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Relational
databases
Non-relational
databases
Data
warehouses
Hadoop
and Spark
Redshif
t
EM
R
Operational
analytics
Elasticsearch
ServiceAuror
a
DynamoD
B
Business
Intelligence
QuickSightRDS
- 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
https://aws.amazon.com/solutions/databasemigrations/database-freedom/
- 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Physical
Data Centers
Wide Platform Support
Any
Application
Any
Database
x86 Operating
Systems
Source
Infrastructure
* See documentation or contact Support for a complete list.
- 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migration Acceleration Program (MAP)
Migration Methodology AWS and Partner Tools AWS Partners
AWS Investment AWS Training AWS Professional
Services
- 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!
wumatt@