The AWS Workshop Series Online is a series of live webinars designed for IT professionals who are looking to leverage the AWS Cloud to build and transform their business, are new to the AWS Cloud or looking to further expand their skills and expertise. In this series, we will cover :'Modern Data Architectures for Business Insights at Scale'.

2. Modern Data Architectures
for Business Insights at Scale
Russell Nash
APAC Solutions Architect

3. Russell Nash
APAC Solutions Architect
Amazon Web Services

4. SCALABLE FLEXIBLE MANAGEABLE
COST
EFFECTIVE
Modern Data Architecture

5. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
Modern Data Architecture
AWS
Cloud Trail
AWS
IAM
Amazon
CloudWatch
AWS
KMS

6. Database Analytics Flat File
Processing
Real-time
Pipeline
Data Lake

7. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
Database Analytics
Amazon
Redshift
Source
Database

8. MPP SQL Database
Optimised for Analytics
Gigabytes to Petabytes
Fully relational
Amazon
Redshift

10. ID Name
1 John Smith
2 Jane Jones
3 Peter Black
4 Pat Partridge
5 Sarah Cyan
6 Brian Snail
1 John Smith
4 Pat Partridge
2 Jane Jones
5 Sarah Cyan
3 Peter Black
6 Brian Snail

11. 1 John Smith
4 Pat Partridge
2 Jane Jones
5 Sarah Cyan
3 Peter Black
6 Brian Snail
SQL
SQL SQL SQLResults Results Results
Results

12. 160 GB
2 PB

13. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
ETL
Amazon
Redshift
Source
Database
Database Analytics

14. AWS Database
Migration Service
Amazon
RedshiftSource
Database
ETL
Data Integration
Partners

15. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
ELT
Amazon
Redshift
Amazon
Redshift
Source
Database
Database Analytics

16. https://aws.amazon.com/solutions/case-studies/boingo-wireless/

17. Database Analytics Flat File
Processing
Real-time
Pipeline
Data Lake

18. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
Batch Processing
Flat
Files
Amazon
S3

19. Amazon
S3
Object Storage
Low Cost
Highly Scalable
11 9’s of durability

20. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
Flat
Files
Amazon
S3
Batch Processing
AWS
Snowball
AWS
CLI & SDK

23. In pioneer days, they used oxen for heavy pulling,
and when one ox couldn’t budge a log,
they didn’t try to grow a bigger ox.
Grace Hopper

25. PIG
Infrastructure
Data Layer
Process Layer
Framework
Applications

26. PIG
SQL
Infrastructure
Data Layer
Process Layer
Framework
Applications

27. PIG
SQL
Amazon
EMR

28. PIG
SQL
Amazon
EMR
Amazon
S3
EMRFS

29. Amazon
EMR
• Managed Hadoop
• Optimized with S3
• Open Source Support

30. Compute Flexibility
Compute Memory Storage
Machine Learning
C4 Family
C3 Family
X1 Family
R3 Family
Interactive Analysis
D2 Family
I2 Family
Large HDFS
General
Batch Process
M4 Family
M3 Family

31. Cost & Time
# CPUs
Time
# CPUs
Time
Wall clock time: 1 hourWall clock time: 10 hours

33. Spot Price – M4.2XL
On-Demand Spot-Price
$0.07$0.53

34. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
Flat
Files
Amazon
S3
Batch Processing
Amazon
EMR
Amazon
S3
AWS
Glue
AWS
Snowball
AWS
CLI & SDK

35. AWS
Glue
• Managed Transform Engine
• Job Scheduler
• Data Catalog
• Built on Apache Spark

37. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
Flat
Files
Amazon
S3
Batch Processing
Amazon
EMR
Amazon
S3
AWS
Glue
Amazon
Redshift
Amazon
EMR
AWS
Snowball
AWS
CLI & SDK

38. PIG
SQL
Amazon
EMR
Amazon
S3
EMRFS
R

39. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
Flat
Files
Amazon
S3
Batch Processing
Amazon
EMR
Amazon
S3
AWS
Glue
Amazon
Redshift
Amazon
EMR
Amazon
AthenaAWS
Snowball
AWS
CLI & SDK

40. Amazon
Athena
Query S3 data with SQL
Serverless
Instant Spin-Up
Pay per Query

41. Athena
S3

42. Comparison of SQL Processing engines
Amazon
Redshift
Amazon
Athena
Data Structure
Languages
Semi Semi
SQL, HiveQL SQL
Full
SQL
Data Store S3/HDFS S3 Local
SQL
Semi
SQL
S3/HDFS
Performance

43. Comparison of SQL Processing engines
Transformation
SQL Queries
For S3/HDFS
Fully Featured
SQL
Database
Use Case
Amazon
RedshiftAmazon
Athena
SQL
Serverless
SQL Queries
for S3

44. https://aws.amazon.com/solutions/case-studies/finra/

45. Database Analytics Flat File
Processing
Real-time
Pipeline
Data Lake

46. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
Real-time Pipeline
Amazon
Kinesis
Machines
Devices
Mobile
Clickstream

47. Availability
Zone
Availability
Zone
Availability
Zone
Amazon Kinesis
Stream
AWS Lambda
KCL App
Amazon EMR
Streaming

48. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
Amazon
Kinesis
AWS Lambda
Application
Amazon EMR
Streaming
S3
(Log)
Amazon
ElasticSearch
(Dashboard)
Real-time Pipeline

49. Amazon
Elasticsearch
• Search and Analytics
• Scalable
• Fully Managed
• Integrated – Logstash, Kibana

51. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
Amazon
Kinesis
AWS Lambda
Application
Amazon EMR
Streaming
S3
(Logs)
Amazon
ElasticSearch
(Dashboards)
Amazon EMR
(Predictions)
ML
Amazon SNS
(Alerts)
Real-time Pipeline
Amazon
Redshift
(Analytics)

52. StreamAlert
https://medium.com/airbnb-engineering/streamalert-real-time-data-analysis-and-alerting-e8619e3e5043

53. Database Analytics Flat File
Processing
Real-time
Pipeline
Data Lake

54. Any data Any analysisData Lake

55. Ingest Serving
Speed (Real-time)
Scale (Batch)
Data analysts
Data scientists
Business users
Engagement platforms
Automation / events
Sources
Amazon
Kinesis
AWS Lambda
Application
Amazon EMR
Streaming
Amazon
EMR
Data Lake
Amazon
Redshift
ETL
Amazon
Athena
EC2
AWS
CLI & SDK
Amazon
S3
Amazon
EMR
Amazon
S3
AWS
Cloud Trail
AWS
IAM
Amazon
CloudWatch
AWS
KMS

56. New X1 Instance - Tons of Memory
• Large-scale, in-memory applications
• Intel® Xeon® E7 8880 v3 Haswell processors
• Up to 2TB of memory
• Up to 128 vCPUs per instance

57. Intel® Processor Technologies
Intel® AVX – Dramatically increases performance for highly parallel HPC workloads
such as life science engineering, data mining, financial analysis, media processing
Intel® AES-NI – Enhances security with new encryption instructions that reduce the
performance penalty associated with encrypting/decrypting data
Intel® Turbo Boost Technology – Increases computing power with performance that
adapts to spikes in workloads
Intel Transactional Synchronization (TSX) Extensions – Enables execution of
transactions that are independent to accelerate throughput
P state & C state control – provides granular performance tuning for cores and sleep
states to improve overall application performance

59. twitter.com/awsawscloudseasia
aws-asean-marketing@amazon.com
facebook.com/amazonwebservices/
youtube.com/user/AmazonWebServices
slideshare.net/amazonwebservices
Thank you for joining us today.
Please complete the survey & let us know what you think of the webinar.

60. Register now for upcoming session for today,
if you would like to join.
https://pages.awscloud.com/aws-workshop-online.html
4pm – 5pm (SGT)
Modern Data Architectures
for Real-time Analytics and Engagement

61. REGISTER NOW
http://amzn.to/2jFt11N
Complimentary labs are available only till 31 March 2017
Get hands on experience working with the AWS Technology.
Access the complimentary Big Data on AWS self-paced labs

62. Q&A