4. 4
4
25 YEARS OF ACCELERATED COMPUTING
X-FACTOR SPEED UP FULL STACK ONE ARCHITECTURE
SYSTEMS
GPU
CPU
5. 5
ANNOUNCING
NVIDIA A100 80GB
Supercharging The Worldâs Highest
Performing AI Supercomputing GPU
80GB HBM2e
For largest datasets
and models
2TB/s +
Worldâs highest memory
bandwidth to feed the worldâs
fastest GPU
Multi-Instance GPU
3rd Gen NVLink
3rd Gen Tensor Core
8. 8
CHALLENGES: ACCELERATING BIG AND SMALL
AI Advances Demand Exponentially
Higher Compute
AI Applications Demand Distributed
Pervasive Acceleration
3000X Higher Compute Required to Train
Largest Models Since Volta
Every AI Powered Interaction Needs
Varying Amount of Compute
AlexNet
ResNet
BERT
GPT-2
Megatron-GPT2
Turing NLG
Megatron-BERT
1E-03
1E-02
1E-01
1E+00
1E+01
1E+02
1E+03
2012 2013 2014 2015 2016 2017 2018 2019 2020 2021
Petaflop/s
-
Days
3000X
10s Billions of Ecom
Recommendations
Billions of
Searches
Millions of
Interactions
Millions of Medical
Scans
Thousands Ads /
Person
Billions of photos
tagged
100s of Billions Events
For Cyber Threat
100s of Millions Fin
Txn For Fraud
AI Interactions Per Day
Source: OpenAI, NVIDIA
9. 9
3 PILLARS OF NVIDIA AI PLATFORM
Pre-Trained Models
SOTA Models
Triton
Inference Serving
Transfer Learning Toolkit
Zero Coding | Speech, Vision & NLU TensorRT 7.2
Conversational AI
Computer Vision
Recommendations
Reinforcement Learning
Frameworks TIS
Query
Result
TLT
Pre-Trained
Model
Data
Customized
Models
TRT
Data Analytics & Training Inference Application Frameworks
CUDA
CUDA-X-AI
HugeCTR
NGC
Merlin
Recommendation Sys
METROPOLIS
Smart City
ISAAC
Robotics
DRIVE
Autonomous Vehicles
CLARA
Healthcare
JARVIS
Conversational AI
Edge Devices Data Center Solutions
Edge Servers Supercomputers
GPU-Accelerated Cloud
10. 10
DIFFERENT SCENARIOS
MLPERF INFERENCE 0.7
mlcommons.org
MANY BENCHMARKS MANY ENVIRONMENTS
Application Network Name
Recommendation
DLRM (99% and 99.9%
accuracy target)
NLP
BERT (99% and 99.9%
accuracy target)
Speech Recognition RNN-T
Medical Imaging
3D U-Net (99% and
99.9% accuracy target)
Image Classification ResNet-50 v1.5
Object Detection
Low Res (Small)
Single-Shot Detector
with MobileNet-v1
Object Detection
High Res (Large)
Single-Shot Detector
with ResNet-34
New
New
New
New
Time-to-Train
To Target Accuracy
On Premise To Cloud
Single Server To Max Scale
MANY CONFIGURATIONS ONE METRIC
11. 11
ENABLING ENTERPRISE TRANSFORMATION WITH AI
End to End Application Frameworks
Desktop Development Data Center Solutions Accelerated Edge Supercomputers GPU-Accelerated Cloud
Jarvis Merlin Metropolis Clara Isaac Drive Aerial
Conversational
AI
Recommender
Systems
Smart Cities Healthcare Robotics Autonomous
Vehicles
Telecom
19. 19
5 DGX A100 systems for AI training
and inference
$1M
1 rack
28 kW
1/10th
COST
1/20th
POWER
$1M 28 kW
DGX A100
DATA CENTER
20. 20
INGESTION STORAGE PROCESSING SERVING
BIG DATA PIPELINE
Ingredients:
⢠Lots of data
⢠Lots of compute
⢠Software tools
⢠Time and patience
Method:
1. Collect raw, massive sets of data.
2. Put the data in a Data Lake.
3. Grab the data that you need and
sort through.
4. Find patterns in the data.
5. Solve the problem.
1. Obtaining and importing
data
2. Organizing & storing data for future use
3. Manipulating and analyzing the
data
4. Operationalizing the
solution
21. RECOMMENDERS â
THE PERSONALIZATION ENGINE OF THE INTERNET
DIGITAL CONTENT
2.7 Billion
Monthly Active Users
E-COMMERCE
2 Billion
Digital Shoppers
SOCIAL MEDIA
3.8 Billion
Active Users
DIGITAL ADVERTISING
4.7 Billion
Internet Users
Item
Candidate
Generation
O(102)
Ranking
User
Embedding
User
Items
Recommende
d
Items
Item
Embedding
O(10)
O(109)
23. 23
HARNESSING
AI
Step I: Build data fabric for your organization
Step II: Define your objective
Step III: Hire the right talent
Step IV: Identify key processes to augment with AI
Step V: Create a sandbox lab environment
Step VI: Operationalize successful pilots
Step VII: Scale up for enterprise-wide adoption
Step VIII: Drive cultural change
24. 24
BRINGING AGILITY TO RETAIL
Real-time speed, predictability, and accuracy are needed
In this highly challenging environment
1. Consumer Behaviour Shifting
2. Supply Chain Under Pressure
3. Real-time Agility Required
Deliver faster forecasting
and replenishment
Decrease labour and cost
challenges
Meet consumer
expectations
Provide omni-channel
convenience
Automate
distribution centres
Ensure employee
and customer safety
25. 25
BUILDING AN AI PRODUCT
SENSORS
PERCEIVE REASON
PLAN
DATA
DATA
ANALYTICS
MACHINE
LEARNING
AI MODEL
VALIDATION
ACTUATORS
AI MODEL
26. 26
Need to Accelerate Data Annotation
⢠Manual Annotation is tedious, time consuming and
expensive
⢠Expert knowledge of Radiologists is required
⢠Most complicated ones are tumors for Brain, Lung, Liver
and Pancreas
⢠Opportunity: AI can accelerate the workflow
(MEDICAL IMAGING) DATA ANNOTATION IS HARD
Time Consuming & Expert knowledge Required
29. 29
Numerous applications
3D DL IS EXCITING
Simulation Medical imaging Autonomous driving
Manipulation Robotics Augmented reality
* This slide is best viewed in "slide show" mode.
30. 30
AI TOOLKITS & SDKs
CONTAINERS TRAINED MODELS HELM CHARTS
100+ 30+ ML, Inference
End-to-End AI Workflows
NGC Catalog
ON-PREM
CLOUD EDGE
HYBRID CLOUD
x86 | ARM | POWER
NGC â ACCELERATING TIME TO SOLUTION
Build AI Faster, Deploy Anywhere
INDUSTRY APP FRAMEWORKS
Curated Software Assets
COLLECTIONS
31. 31
NGC COLLECTIONS
https://ngc.nvidia.com
Search NGC for
the app or use
case
Deploy container Start Jupyter Notebook
instance
Download model
Fine-tune and
deploy
The Collection walks through
all the required assets and
how to use them together
Ready To Use Collections
Conversational AI | Computer Vision | NVIDIA AI App Frameworks
32. 32
GET STARTED WITH NGC
Deploy containers:
ngc.nvidia.com
Learn more about NGC offering:
nvidia.com/ngc
Technical information:
developer.nvidia.com
Explore the NGC Registry for DL, ML & HPC
33. 33
World Sense See, Understand Automation
AI Program
Computer
ARTIFICIAL INTELLIGENCE IS DOMAIN SPECIFIC
Self-Driving
34. 34
World Sense See, Understand Automation
AI Program
Computer
AI Program
Computer
ARTIFICIAL INTELLIGENCE IS DOMAIN SPECIFIC
Self-Driving
Manufacturing
35. 35
World Sense See, Understand Automation
AI Program
Computer
AI Program
Computer
AI Program
Computer
ARTIFICIAL INTELLIGENCE IS DOMAIN SPECIFIC
Self-Driving
Manufacturing
Radiology
37. 37
INTELLIGENT SUPPLY CHAIN
LAST MILE DELIVERY
Last-mile-delivery
INTELLIGENT WAREHOUSES
Loading Dock Intelligence and Data Capture
Package Lifecycle Tracking
Adaptive Speed Conveyor
Smart Cabinets
FORECASTING
Increased Speed and Accuracy
of Forecasting
38. 38
INTELLIGENT
DISTRIBUTION CENTERS
⪠Fully automated multi-shuttle distribution
center leveraging AI, ML, data science,
cloud
⪠Intelligent structures with robotic
carts, arms, and IoT at the edge
⪠Delivering expedited through-
put and improved order
accuracy
39. 39
AI is changing the online shopping
experience with improvements to both
retailers and consumers
Online grocery giant Ocado has already
improved customer service with its AI-
enhanced contact center and is now
applying GPU-powered machine learning
and computer vision to replace
barcode systems, expediting the
picking process and improving
order accuracy
DELIVERING A BETTER
CUSTOMER EXPERIENCE
40. 40
AI HELPS DOMINOâS PREDICT WHEN
THREE BILLION PIZZAS ARE READY
FOR PICKUP
⪠Dominoâs has over 17,000 stores worldwide and
delivers more than 3 billion pizzas a year
⪠AI is used to improve operational efficiencies and
enhance the customer experience including:
- predicting order readiness
- curbside pickup
- routing deliveries
- determining ideal timing for marketing
campaigns
⪠Leveraging NVIDIA DGX-1 servers and
NVIDIA RAPIDS data science libraries
to power their data science platform
⪠Model training time has
improved 72x and order
readiness prediction has
improved from 75% to 95%
41. 41
AUTOMATED PRODUCT META-
TAGGING FOR GREATER
ACCURACY
⪠Retailers use computer vision, NLP and
image recognition to automatically
generate meta-tagging and cataloging
to refine and enrich their data
⪠Comprehensive product and services
meta data helps retailers develop
more powerful personalized
recommendation systems
42. 42
NVIDIA EGX EDGE COMPUTING
.5 TOPS
COMPUTE & AI BY NVIDIA
NETWORK, STORAGE, SECURITY BY MELLANOX
10,000 TOPS
520 TOPS
A new class of distributed AI computing systems
designed to gather and analyze continuous streams
of data at the edge of the network.
AI computation is performed largely or completely
on the EGX systems close to the data or user.
NVIDIA EGX is for applications that require:
Low-latency interactions
Reduced bandwidth to the cloud
Data privacy or sovereignty
44. 44
The human resource risk in distribution and supply chain centers in terms of
worker exposure to COVID-19 is significant and disruptive to operations.
Ipsotek, an NVIDIA Metropolis partner repurposes existing E-commerce
warehouse security cameras to alert for social distancing risk factors and
generates key operational metrics covering worker interactions.
The deployments have resulted in improved occupancy and safe-distancing
procedures along with redesigned spaces.
Initiative reduced risk factors by over 70% and improved space utilization
with non-intrusive technology
E-COMMERCE USES METROPOLIS TO REDUCE
COVID RISK TO SUPPLY CHAIN OPERATIONS
52. 52
DEPLOY ON ANY NVIDIA RTX GPU
From laptop, to data center. On-prem, or in the cloud.
NVIDIA Studio
Any RTX Workstation or Laptop
EGX Platform
NVIDIA Certified System with
RTX
GeForce RTX
GeForce RTX 2060 to
GeForce RTX 3090
Professional Visualization
Quadro RTX 4000 to
NVIDIA RTX A6000
53. 53
CUTTING EDGE APPLICATIONS
Core Omniverse Apps
FOR ARCHITECTS, DESIGNERS, ENGINEERS FOR ROBOTICISTS, SIMULATION SPECIALISTS
FOR GEFORCE RTX GAMERS
FOR DESIGNERS, CREATORS, ENGINEERS FOR 3D DEEP LEARNING RESEARCHERS
FOR GAME DEVELOPERS, ANIMATORS
54. 54
Anything that Moves will be Autonomous
Autonomous machines excel in maximum
efficiency and accuracy from product
assembling, to warehouses management.
Anything that is Built will be Visualized
True-to-reality simulation achieves faster
time-to-production with higher build quality
than purely physically-based prototypes
POWER LARGE SCALE SIMULATION
Physically accurate, complex virtual worlds
Anything Autonomous will be Simulated
AI agents can only achieve intelligence by
training in scalable, photorealistic
environments that obey the laws of physics.
70. 70
CONVERSATIONAL AI IS TRANSFORMING INDUSTRIES
IN-CAR ASSISTANTS
75M New Cars per Year
Alex: Find us a Mexican restaurant
Jarvis: The nearest Mexican
Restaurant is Luna
Kitchen
VIDEOCONFERENCE
CC, TRANSLATION, TRANSCRIPTION
200M Meetings per Day
SMART SPEAKERS
150M Sold per Year
RETAIL ASSISTANTS
12M Retail Stores
CALL CENTER
500M Calls per Day
+
71. ANNOUNCING
JARVIS OPEN BETA
Integrated AI Skills with Pre-Trained Models
Fully Customizable Application Pipeline
Human Voice with Neural TTS
Superhuman NLU with Megatron-BERT
<300 ms Latency | 7X Throughput | 1/3rd Cost
Sign Up at developer.nvidia.com/nvidia-jarvis
State-of-the-Art Conversational AI
74. 74
NVIDIA EGX METROPOLIS BENEFITS
Largest domain of AI models
High throughput & low latency
One architecture scalable from device
to cloud
NVIDIA is Industryâs Most
Advanced AI Computing Platform
Hardware and software optimized for
AI, storage, networking & security
Easy development & deployment of AI
at edge
Optimized AI models in NGC
NVIDIA EGX is AI-Optimized
Hyper-converged Infrastructure
Support for every platform â VMW, RH,
NTNX, Azure, AWS, GCP
Rich 3rd party ISV ecosystem
Rich OEM and integrator ecosystem
NVIDIA is an
Open Platform
Every cloud
Hybrid cloud
Edge to cloud
NVIDIA is Pervasive
AI Platform
End-to-end, from development to
deployment, from tools to experts
NVIDIA Research
DLI to reskill talent
SA & DevTech to co-engineer
NVIDIA has
Deep AI Expertise
Expertise in large verticals â M&E,
healthcare, retail, manufacturing,
transportation, and more
BD, SA, DevRel, DevTech, Research in
every region
NVIDIA has Global Reach
and Support
75. 75
RICH
CONTENT
PORTFOLIO
Fundamentals and advanced
hands-on training in key
technologies and application
domains
AI for
Digital Content Creation
Deep Learning
Fundamentals
AI for Healthcare
AI for Autonomous Vehicles
AI for
Intelligent Video Analytics
Accelerated Computing
Fundamentals
AI for Robotics
AI for
Predictive Maintenance
Accelerated Data Science
Fundamentals
Intro to AI in the Data
Center
AI for Anomaly Detection
AI for Industrial Inspection
76. 76
EDUCATION AND
TRAINING
Coding, getting started, and how-to demos
Talks by NVIDIA developer and engineers
Event keynotes, sessions, and more
Playlists organized by topics
Community contributions
YouTube Channels
www.youtube.com/user/NVIDIADeveloper
77. NVIDIA INCEPTION
ACCELERATING 6K STARTUPS WORLDWIDE
EXPERTISE
NVIDIA Deep Learning Institute
Training in AI, accelerated computing, and
accelerated data science
TECHNOLOGY ASSISTANCE
Developer resources, preferred pricing on on-prem
GPUs, and cloud credits through our global partners
GO-TO-MARKET SUPPORT
Networking events and exposure opportunities
through NVIDIA
VENTURE CAPITAL FUNDING & ECOSYSTEM
NVIDIA Inception GPU Ventures
Investing in breakthrough startups and facilitating
engagements with the VC community
www.nvidia.com/inception
78. NVIDIAâs GTC brings together a global community of
developers, researchers, engineers, and innovators
to experience global innovation and collaboration.
Donât miss out on the exclusive GTC keynote by Jensen Huang
on April 12, available to everyone.
Visit www.nvidia.com/gtc to learn more and be notified when
registration opens.
THE CONFERENCE FOR AI INNOVATORS,
TECHNOLOGISTS, AND CREATIVES
Join us at GTC 2021 on April 12 - 16 for the latest
in AI, HPC, healthcare, game development,
networking, and more.