Marv Wexler - Transform Your with AI.pdf

Confidential
Transform Your
Business With AI
Transform Your
Business With AI
AI Summit
Marv Wexler
GM Technical Services
September 21, 2023
AI Summit
Marv Wexler
GM Technical Services
September 21, 2023
Better Faster Greener™ © 2023 Supermicro

Confidential
Where are we on the AI journey ?
9/20/2023 Better Faster Greener™ © 2023 Supermicro
2
“Once a new technology rolls over you, if you're not part of the steamroller, you're
part of the road.” - Stewart Brand

Confidential
3
Current AI Trends
• Democratization of AI will continue
• AI is a fundamental differentiator for businesses
• Find deeper insights in data, real-time and at scale
-Else your competitors surely will
• Generative AI is becoming commercialized
• AI ethics a top priority
• Biased algorithms, Deep fakes, “Hallucinations” as a
feature
• Generative AI applications reign : Microsoft (Designer),
Adobe (Firefly), Meta (Ad creation)
• New regulations for safe and responsible practices
• EU AI Act: Set of new rules that establish obligations for risks
from artificial intelligence

Confidential
AI Applications
4
Deep Learning
Solving complex
problems
Computer model taught to
learn actions using images,
texts and sounds
Machine Learning
Machines making
decisions
Building Machines with
predictive algorithm and
create predictive models
Artificial Intelligence
Simulate intelligence
Building Smart Machines
capable of performing
intelligent tasks

Confidential
5
Text
Image
Audio
Video
Games
Text/ Voice prompt
Generative AI models
(also Large Language
LLM, or Foundational
Models)
User Input
What is Generative AI?
Generative AI models are models that, when receiving a text prompt, give an output related to
that input. The output can be text, image, audio, video, code etc.
The ability for generative AI to produce useful, impressively synthesized text, images, and other types of content
almost effortlessly based on a few text cues has already become an important business capability worthy of
providing immense value to most knowledge workers

Confidential
The far-reaching impacts of Generative AI
6
Around 75% of the technology's value will be seen across four areas:
• customer operations
• marketing and sales
• software engineering
• research and development
automating conversations with customers
creating personalized messages for customers
generating code
generative design

Confidential
Customizable AI infrastructure for Generative AI
7
Training
•compute intensive
•massive datasets
involved
Fine-Tuning
•Requires relatively less
computational power
Inferencing
•Accelerators may be
needed depending on
type of application
(batch/real-time)
Various stages in building a
Generative AI Application
At Supermicro, We have you covered all the way with affordable, customizable
and scalable solutions

Confidential
Application Details
8
LangChain Instructor Embeddings WizardLM / LLAMA
• Ask questions to your documents AND learn from your documents using the
power of LLMs.
• 100% private, no data leaves your execution environment at any point.
• You can ingest documents and ask questions without an internet connection!
localGPT
BUILT WITH
• Text pre processed
into chunks
• Embedded in a
vector space
• Query search for
similar chunks
An instruction-finetuned text
embedding model that can generate
text embeddings tailored to any
task by simply providing the task
instruction, without any finetuning.
Instructor achieves SOTA on 70
diverse embedding tasks!
(e.g., classification, retrieval,
clustering, text evaluation, etc.) and
domains (e.g., science, finance, etc.)
• WizardLM is a Llama variant
trained with
complex instructions
• Evol-Instruct which
leverages AI to
"evolve" instructions

Confidential
Application Details
9
Ingest.py
• uses LangChain tools to parse the document and create
embeddings locally using Instructor Embeddings
Chroma
vector store
• local vector database that stores the created
embeddings
Run_localGPT • uses local LLM to understand questions and create
answers.
Similarity
Search
• used to extract right piece of context
from the local vector store

Confidential
10
©2023 Supermicro
Large Scale AI Training
• Key Technologies
• NVIDIA HGX H100 SXM 8-GPU/4-GPU with 900GB/s NVLink interconnect
• Dedicated, lots of high performance, high bandwidth GPU memory - HBM3, HBM2e
• 400GbE networking (Ethernet or InfiniBand), PCIe 5.0 storage for fast AI data pipe
• NVIDIA GPUDirect RDMA and Storage to keep feeding data to GPUs with minimum latency
• Liquid cooling for GPUs and CPUs
• All-flash storage and file systems to support petabytes of hot-tier data cache
• NVIDIA HGX H100 SXM5
board with 4- GPU or 8-
GPU
• NVLink and NVSwitch
• 80GB HBM3 per GPU
• Up to 700W TDP
• NVIDIA ConnectX-7
• Up to 400GbE or 400G NDR InfiniBand
• x16/x32 PCIe 5.0

Confidential
Supermicro AI
Experience
Supermicro AI
Experience
Marv Wexler
August 2023
Marv Wexler
August 2023
Better Faster Greener™ © 2023 Supermicro

Confidential
12

Confidential
Evolving to an AI / Total IT Solutions Partner
13
 5S: Software, Services,
Switch, Storage, Security
and more
 Total Solutions: Enterprise,
OEM- Appliance / Cloud
 Complete Systems
 Sub-systems and
Components
~5X+ Faster growth rate than the
industry avg rate over the past 12+
months (~50% YoY)
~5X+ Faster growth rate than the
industry avg rate over the past 12+
months (~50% YoY)
Our Momentum:
SMCI 1.0
Components &
Subsystems
SMCI 2.0
Servers &
Storage Systems
SMCI 3.0
Total IT
Solutions
Today
1993
$5B
$10B

Confidential
SMCI AI Strategy
14
• Partner with the Leaders
• Provide the best picks and shovels for the gold miners (Apps, YOU)
• Do not be religious with Products Offerings (multi-vendor, multi-platform)

Confidential
SMCI AI Business Results
15
• Bring up platform partner for virtually all AI Solutions / GPU offerings
• Lead supplier for virtually all Large Language Model Cloud Deployments
(ChatGPT, BARD, Bing, etc.)
The Next Platform, August 16, 2023

Confidential
16
©2023 Supermicro
GPU Optimized Systems by Workloads
• Large Scale AI Training • HPC/AI Workloads
H100 PCIe
Grace Hopper Superchip (Grace
CPU + H100 GPU)
H100 NVL
HGX H100 SXM
8-GPU or 4-GPU
4U 4-GPU System (HGX H100 SXM)
(codenamed: Redstone-Next)
SYS-421GU-TNXR, SYS-521GU-TNXR
(codenamed: Delta-Next)
SYS-821GE-TNHR, AS -8125GS-TNHR
SYS-421GU-TNXR
4U/5U 8-10 GPU System
SYS-521GE-TNRT, SYS-421GE-TNRT/TNRT3
AS -4125GS-TNRT/TNRT1/TNRT2
1U Grace Hopper MGX System
SYS-421GU-TNXR / SYS-521GU-TNXR
8U SuperBlade (Up to 20 nodes)
SBI-411E-1G / SBI-411E-5G
Petabyte Scale All-Flash Storage
SSG-121E-NE316R, ASG-1115S-NE316R

Confidential
Scales to thousands of nodes in 32-node increments
(SRS-42UHPC-32SU-01)
Accelerate AI Development by Supermicro
Supermicro 8U Delta-Next (SYS-821GE-TNHR)
A Proven Platform, Purpose Built for AI
H100 SXM5 GPU ConnectX-7 SmartNICs
H100 Rack Scale SuperPod Scalable Unit
8x NVIDIA H100 SXM5 GPUs | 640GB HBM3 GPU Memory 2TB
System Memory | 3.2Tbps Network B/W | Superior I/O
32x HGX H100 | 1+ EFLOPS AI | 20TB HBM3 GPU Memory 102.4Tbps
Network B/W Non-blocking | InfiniBand NDR
Software: NVIDIA BCM | NGC | NVAIE | SLURM | Kubernetes
Full Turnkey AI Supercomputer for Enterprises
17

Confidential
Supermicro Rack Integration Services
• Full rack integration up to L11 and L12
• Broad portfolio of compute, power, cooling
and networking options
• Liquid cooling integration
• Cooling Distribution Unit (CDU)
• Direct to Chip cold plate
• Manifold and tubing
• Design, assembly, configuration, testing
and deployment
• Start running applications from Day 1

Confidential
Supermicro CDU
80kW to 120kW, 45°C Warm Water
Liquid Cooling Option for Rack Scale H100 SuperPods
19

Confidential
Onsite Rack Services
20
Simplifying Your Solution Deployment Needs
• White glove custom service from beginning to end
• Onsite rack & stack of the custom solution
• Onsite integration ensuring proper installation and
connectivity, providing for reliable operation and reduced
downtime
• Onsite software installation with application configurations
• Onsite benchmark testing ensuring solution meets the
requirements of the customer
• Delivery of a customized rack solution that meets all
requirements
• SMC Cooling tower product line is available to enable
facility level water connections for CDU/CDM/RDHX
Reliable – Repeatable – Reproducible

Confidential
DISCLAIMER
Super Micro Computer, Inc. may make changes to specifications and product descriptions at any time, without notice. The
information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions
and typographical errors. Any performance tests and ratings are measured using systems that reflect the approximate
performance of Super Micro Computer, Inc. products as measured by those tests. Any differences in software or hardware
configuration may affect actual performance, and Super Micro Computer, Inc. does not control the design or implementation of
third party benchmarks or websites referenced in this document. The information contained herein is subject to change and may
be rendered inaccurate for many reasons, including but not limited to any changes in product and/or roadmap, component and
hardware revision changes, new model and/or product releases, software changes, firmware changes, or the like. Super Micro
Computer, Inc. assumes no obligation to update or otherwise correct or revise this information.
SUPER MICRO COMPUTER, INC. MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE
CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT
MAY APPEAR IN THIS INFORMATION.
SUPER MICRO COMPUTER, INC. SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR
FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL SUPER MICRO COMPUTER, INC. BE LIABLE TO ANY
PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF
ANY INFORMATION CONTAINED HEREIN, EVEN IF SUPER MICRO COMPUTER, Inc. IS EXPRESSLY ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
ATTRIBUTION
© 2023 Super Micro Computer, Inc. All rights reserved.
21

Confidential
www.supermicro.com

Marv Wexler - Transform Your with AI.pdf

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Marv Wexler - Transform Your with AI.pdf

Similar to Marv Wexler - Transform Your with AI.pdf (20)

More from SOLTUIONSpeople, THINKubators, THINKathons

More from SOLTUIONSpeople, THINKubators, THINKathons (12)

Recently uploaded

Recently uploaded (20)

Marv Wexler - Transform Your with AI.pdf