인텔 기술을 활용한 AI의 비용 대비 성능 극대화 - Eitan Medina Chief Business Officer, Intel :: AWS Summit Seoul 2021

K O R E A | M A Y 1 1 - 1 2 , 2 0 2 1

© 2021, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Akanksha Balani
AWS APJ Alliance head @ Intel
Global AI GTM Lead
Unleash price-performance
advantage for AI on Intel

AI Today
Why Intel & AWS
Intel AI on AWS
Introducing Habana – Gaudi instance
Learn | Engage | Innovate AI with Intel
Agenda

Source: Intel forecast
Consumer
Smart
Assistants
Chatbots
Search
Personalization
Augmented
Reality
Robots
Health
Enhanced
Diagnostics
Drug
Discovery
Patient Care
Research
Sensory
Aids
Finance
Algorithmic
Trading
Fraud Detection
Research
Personal
Finance
Risk Mitigation
Retail
Support
Experience
Marketing
Merchandising
Loyalty
Supply Chain
Security
Government
Defense
Data
Insights
Safety &
Security
Resident
Engagement
Smarter
Cities
Energy
Oil & Gas
Exploration
Smart
Grid
Operational
Improvement
Conservation
Transport
Autonomous
Cars
Automated
Trucking
Aerospace
Shipping
Search &
Rescue
Industrial
Factory
Automation
Predictive
Maintenance
Precision
Agriculture
Field Automation
Other
Advertising
Education
Gaming
Professional &
IT Services
Telco/Media
Sports
Transformation with AI

Platforms Finance Healthcare Energy Industrial Transport Retail Home More…
Toolkits
App Developers
Libraries
Data Scientists
Foundation
Library Developers
Hardware
IT System Architects
Solutions
Solution Architects
OpenVINO™ Toolkit Deep Learning Developer Toolkit
OpenVINO Toolkit for inference deployment on CPU, processor
graphics, FPGA & VPU using TF, Caffe* & MXNet*
Optimized inference deployment for all Intel® Movidius™ VPUs
using TensorFlow* & Caffe*
DEEP LEARNING FRAMEWORKS
TensorFlow* MXNet* Caffe* BigDL/Spark* Caffe2* PyTorch* PaddlePaddle*
MACHINE LEARNING LIBRARIES
Python
• Scikit-learn
• Pandas
• NumPy
R
• Cart
• RandomForest
• E1071
Distributed
• MlLib (on Spark)
• Mahout
ANALYTICS, MACHINE & DEEP LEARNING PRIMITIVES
Python DAAL MKL-DNN clDNN
Intel distribution optimized for
machine learning
Intel® Data Analytics Acceleration Library
(for machine learning)
Open-source deep neural network functions for
CPU, processor graphics
† Formerly the Intel® Computer Vision SDK
*Other names and brands may be claimed as the property of others.
All products, computer systems, dates, and figures are preliminary based on current expectations, and are subject to change without notice.
Inference
Artificial Intelligence
DEEP LEARNING
ACCELERATORS
AI FOUNDATION

What does Intel do with Amazon?
COMMON HISTORY AND VALUES
14+ year engineering partnership
Digital transformation
Shared customer passion
High performance + low costs
World-class supply chain
Amazon EC2
VMware
Cloud on
AWS
AWS IoT
Core
AWS
Greengrass
Amazon
SageMaker
AWS Deep
Learning
AMIs
AWS
DeepLens
Amazon S3
AWS
Deepracer
Amazon
Echo Look
Amazon
Echo Show
Cloud &
data center
Things &
devices
Alexa Voice
Services
SAP on AWS
AWS Outposts
Intel is a very deep partner
of AWS and will be for a long
time. That’s not changing.
Andy
Jassy,
CEO, AWS

Greatest variety and availability to meet your
global workload needs
250+ Intel instances
14 years of partnership
2017 2021
Memory optimized
R5 | R5n | R5b | X1e / X1 | High Memory | Z1d
Accelerated compute
Gaudi Instances | P3 | G4 | F1
General purpose
T3 | M5 | M5n | M5zn | M5dn
Compute optimized
C5 | C5n | C5d | C5dn
Storage optimized
I3 | I3en | D3/D3en

Intel-based EC2 Instances for ML
More info: https://aws.amazon.com/ec2/instance-types/
R5 instances are for memory intensive
workloads which use 3D-CNN/BERT-
large/T5 topologies with memory
requirement more than 192GB.
T3
Burstable
General Purpose
M5n
M5d
n
Memory-intensive
X1
X1e
Large
Memory
I3en
Compute-intensive
Z1d
M5
M5d
Skylake/Cascade lake Haswell
HM
C5
High Frequency
High I/O
R5
R5d
R5n
R5d
n
C5n
C5d
T3 instances are better suitable ML
applications and low compute DL
inference applications.
M5, C5 instances are suitable for all
Computer Visions, ML and DL inference
workloads
2nd Generation Intel Xeon Scalable Processor
DL Boost for Inference, single node training
C5n instances are suitable for Distributed
Deep learning training due to high NW
performance required for inter-node
communication.
Baremetal instances are preferred for
large topologies as HVM based instances
adds ~10% performance overhead.

AWS Machine Learning Acceleration with Intel
AWS C5 Deep Learning AMI
Optimized for Intel CPU
(Training)
AWS C5 Deep Learning AMI
(Inference)
AWS SageMaker Machine Learning
(Training & Inference)
faster than training on the stock
Tensorflow 1.6 binaries
7.4x 12x
faster than default configuration for NMT
Inference with MxNet
10x
Machine Learning Algorithms optimized for
IA CPU
AWS MARKETPLACE: Deep learning AMIs | Multiple containers with OneDNN-
optimized frameworks
Amazon Personalize
Amazon Rekognition
Amazon SageMaker Amazon Forecast

Industry
Success Stories
Business value
AI / ML customer programs with Intel
Training
Support
Co-
Engineer
Support
Joint
Pilots &
POCs
Customer
s
HOW TO ENGAGE
Participate in an Intel/AWS DevJam at the AWS Summits – NLP based challenge on Intel/AWS
Explore 16 Intel-optimized software libraries and frameworks on AWS Marketplace
Learn from other customers like FootAsylum, Thorn, Zignal Labs, Krispy Kreme, GE, Thomson
Reuters, ASIC: https://aws.amazon.com/solutions/case-studies/
Connect with a partner – DataRobot, Data Bricks, IntellHQ, Intellify, Peak.AI, C3.AI, H2O.AI, Slalom
RESULTS

AI Success Stories
AI solution with the Intel®
Distribution of OpenVINO™ toolkit
Helps identify, detect, and respond in real
time to hazards along Singapore’s coastline.
Researchers democratized data and drove
ML models on genomic sequencing for
endangered species.
Worked with educators to address the needs
of Delhi’s schools during Covid-19. Within
two months a solution was was scaled to
serve over 160,000 students.
Delivered more than 100 dashboards using
anonymized government and public COVID-
19 data points to prevent disease
transmission.
Australia’s CSIRO research agency was able
to increase by 60x the number of parallel
wildfire simulations with 98% utilization of
large Amazon EC2 C5-based instances.
Analyzed 3 TB of Plant Breeding and
Acclimatization Genomics Data in Hours to
study the diversity of wheat.

Thorn Helps Law Enforcement Identify Child Sex-Trafficking
Victims Faster on AWS & Intel
PROJECT:
The National Center for Missing and Exploited Children
(NCMEC) has about 460,000 images of missing
children in their database.
Using the compute power of Intel® Xeon® Scalable
processors in C5, Thorn is able to match thousands of
faces per second to identify trafficking victims.
COLLABORATION:
• Thorn and AWS-partner, MemSQL, built a facial
recognition service using data analytics and deep
learning on C5 to identify trafficking victims.
• Thorn can apply data points to a single face and
classify, correlate, and match the image to an image
in a database. As a result, the organization’s
solution can make a positive image match in
seconds, instead of minutes.
• Spotlight identifies an average of 5 kids per day.

Introducing Habana Gaudi-based Instances
ML TRAINING POWERED BY NEW HABANA GAUDI PROCESSORS FROM INTEL
COMING IN 2021
New EC2 instances built
specifically for ML training and
powered by up to 8 new Habana
Gaudi processors from Intel
Will deliver up to 40% lower cost-
to-train deep learning models
over GPU-based instances
Will allow customers to iterate
and train models more frequently
Benefit from full stack of Amazon
EC2 services—DL AMIs, DLC for
containerized applications,
ultimately SageMaker
Developers can implement
Gaudi-based instances via AWS
ECS and EKS for containerized
applications
Will support common frameworks
like TensorFlow and PyTorch
Wide range of ML workloads for
applications including, NLP,
image classification, object
detection, recommendation
systems
For efficient scaling across
multiple Gaudi-based EC2
Instances, support for AWS
Elastic Fabric Adapter

Introducing the GAUDI® AI Processor:
powering better price-performance in
upcoming Amazon EC2 Instances
Eitan Medina
CBO, Habana Labs

Founded in 2016
• To design & build processors expressly for AI data center and cloud
Acquired January 2020
• Operates as an autonomous BU reporting to Intel Data Platform Group
• Controls roadmap, R&D, HR, OPS and supply chain
• Former Intel AIPG employees moved into Habana
Benefits from Intel’s resources, R&D scale and relationships
Habana, an Intel Company

Product lines optimized for AI functions
Inference Processors
Training Processors

Gaudi-powered
Amazon EC2 Instances

▪ Amazon’s first non-GPU EC2
instances
▪ Based on Habana GAUDI
Processors,
custom built for AI Training
▪ According to AWS at re:Invent 2020:
Gaudi-based EC2
Instances coming in 1H,
2021
“The new EC2 instances will leverage
up to 8 Gaudi accelerators and deliver
up to 40% better price/performance
than current GPU-based EC2
instances for training DL models.”

Designed for AI performance, flexibility and scale
Habana reports extraordinary training performance …
For ResNet50 on Tensorflow, 8-Gaudi system delivers
Gaudi lowers time and cost-to-train
12,008
Images/second
throughput

▪ EC2 Instances will feature up to 8 accelerators
▪ Benefit from full stack of Amazon EC2 services—DL AMIs, DLC for containerized
applications, ultimately SageMaker
▪ Developers can implement Gaudi-based instances via AWS ECS and EKS for
containerized applications
▪ Will support a wide range of ML workloads for applications including, NLP, image
classification, object detection, recommendation systems
▪ For efficient scaling across multiple Gaudi-based EC2 Instances, support for
AWS Elastic Fabric Adapter
Gaudi-based Amazon EC2 Instances

Architected for flexibility
and scale
▪ Fully programmable, 8 Tensor
Processing Cores (TPC) and
configurable Matrix Math engine
(GEMM)
▪ Multi-stage memory hierarchy with
32GB HBM2 memory
▪ Delivers higher efficiency than
traditional CPUs and GPUs
▪ 10 x 100 Gigabit Ethernet for multi-chip
scale-out training
Custom built for training Deep Learning models

▪ Train deep learning models on Gaudi
with minimal code changes
▪ Natively integrated with TensorFlow
& PyTorch
▪ Reference models, kernel libraries,
software and docs available on
GitHub
▪ Advanced users can write their own
custom software kernels
▪ Software will be pre-integrated in
Amazon Deep Learning AMI and
Containers
Gaudi SynapseAI® Software Suite in EC2
Graph Compiler
Habana Communication
Libraries
Habana
Kernel
Library
Custom
Kernel
Library
User Mode Driver
Kernel Mode Driver
Debugging
&
Profiling
Tools
TPC
Programming
Tools
Framework Integration Layer

▪ Lower cost-to-train and time-to-train
▪ Lower total cost of operations for customers
▪ Enabling more customers to leverage business….
▪ Insights
▪ Efficiencies
▪ Enhanced end-user experiences
…that AI training can provide.
Gaudi-based EC2 Instances will help…

Thank you!

More information at
https://aws.amazon.com/intel/
on AWS & Intel
New instances based on Intel Xeon
Scalable on AWS (M5, C5, C5n, R5, T3)
Contact your Intel/AWS representative for
access to Intel AI and POC opportunities/case
studies
Learn
Explore
Engag
e
CREATE WORLD CHANGING TECHNOLOGY THAT ENRICHES THE LIVES OF EVERY PERSON ON EARTH
Learn | Explore | Engage AI on Intel

여러분의 소중한 피드백을 기다립니다.
강연 종료 후, 강연 평가에 참여해 주세요!

감사합니다

인텔 기술을 활용한 AI의 비용 대비 성능 극대화 - Eitan Medina Chief Business Officer, Intel :: AWS Summit Seoul 2021

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to 인텔 기술을 활용한 AI의 비용 대비 성능 극대화 - Eitan Medina Chief Business Officer, Intel :: AWS Summit Seoul 2021

Similar to 인텔 기술을 활용한 AI의 비용 대비 성능 극대화 - Eitan Medina Chief Business Officer, Intel :: AWS Summit Seoul 2021 (20)

More from Amazon Web Services Korea

More from Amazon Web Services Korea (20)

Recently uploaded

Recently uploaded (20)

인텔 기술을 활용한 AI의 비용 대비 성능 극대화 - Eitan Medina Chief Business Officer, Intel :: AWS Summit Seoul 2021