Marv Wexler - Transform Your with AI.pdf

SOLTUIONSpeople, THINKubators, THINKathons
SOLTUIONSpeople, THINKubators, THINKathonsThinkubator đź’ˇInnovation Experience Designer đź’ˇ Linkedin's #1 Most Connected Innovator in The World
••
Confidential
Transform Your
Business With AI
Transform Your
Business With AI
AI Summit
Marv Wexler
GM Technical Services
September 21, 2023
AI Summit
Marv Wexler
GM Technical Services
September 21, 2023
Better Faster Greener™ © 2023 Supermicro
Confidential
Where are we on the AI journey ?
9/20/2023 Better Faster Greener™ © 2023 Supermicro
2
“Once a new technology rolls over you, if you're not part of the steamroller, you're
part of the road.” - Stewart Brand
Confidential
9/20/2023 Better Faster Greener™ © 2023 Supermicro
3
Current AI Trends
• Democratization of AI will continue
• AI is a fundamental differentiator for businesses
• Find deeper insights in data, real-time and at scale
-Else your competitors surely will
• Generative AI is becoming commercialized
• AI ethics a top priority
• Biased algorithms, Deep fakes, “Hallucinations” as a
feature
• Generative AI applications reign : Microsoft (Designer),
Adobe (Firefly), Meta (Ad creation)
• New regulations for safe and responsible practices
• EU AI Act: Set of new rules that establish obligations for risks
from artificial intelligence
Confidential
AI Applications
9/20/2023 Better Faster Greener™ © 2023 Supermicro
4
Deep Learning
Solving complex
problems
Computer model taught to
learn actions using images,
texts and sounds
Machine Learning
Machines making
decisions
Building Machines with
predictive algorithm and
create predictive models
Artificial Intelligence
Simulate intelligence
Building Smart Machines
capable of performing
intelligent tasks
Confidential
9/20/2023 Better Faster Greener™ © 2023 Supermicro
5
Text
Image
Audio
Video
Games
Text/ Voice prompt
Generative AI models
(also Large Language
LLM, or Foundational
Models)
User Input
What is Generative AI?
Generative AI models are models that, when receiving a text prompt, give an output related to
that input. The output can be text, image, audio, video, code etc.
The ability for generative AI to produce useful, impressively synthesized text, images, and other types of content
almost effortlessly based on a few text cues has already become an important business capability worthy of
providing immense value to most knowledge workers
Confidential
The far-reaching impacts of Generative AI
9/20/2023 Better Faster Greener™ © 2023 Supermicro
6
Around 75% of the technology's value will be seen across four areas:
• customer operations
• marketing and sales
• software engineering
• research and development
automating conversations with customers
creating personalized messages for customers
generating code
generative design
Confidential
Customizable AI infrastructure for Generative AI
9/20/2023 Better Faster Greener™ © 2023 Supermicro
7
Training
•compute intensive
•massive datasets
involved
Fine-Tuning
•Requires relatively less
computational power
Inferencing
•Accelerators may be
needed depending on
type of application
(batch/real-time)
Various stages in building a
Generative AI Application
At Supermicro, We have you covered all the way with affordable, customizable
and scalable solutions
Confidential
Application Details
9/20/2023 Better Faster Greener™ © 2023 Supermicro
8
LangChain Instructor Embeddings WizardLM / LLAMA
• Ask questions to your documents AND learn from your documents using the
power of LLMs.
• 100% private, no data leaves your execution environment at any point.
• You can ingest documents and ask questions without an internet connection!
localGPT
BUILT WITH
• Text pre processed
into chunks
• Embedded in a
vector space
• Query search for
similar chunks
An instruction-finetuned text
embedding model that can generate
text embeddings tailored to any
task by simply providing the task
instruction, without any finetuning.
Instructor achieves SOTA on 70
diverse embedding tasks!
(e.g., classification, retrieval,
clustering, text evaluation, etc.) and
domains (e.g., science, finance, etc.)
• WizardLM is a Llama variant
trained with
complex instructions
• Evol-Instruct which
leverages AI to
"evolve" instructions
Confidential
Application Details
9/20/2023 Better Faster Greener™ © 2023 Supermicro
9
Ingest.py
• uses LangChain tools to parse the document and create
embeddings locally using Instructor Embeddings
Chroma
vector store
• local vector database that stores the created
embeddings
Run_localGPT • uses local LLM to understand questions and create
answers.
Similarity
Search
• used to extract right piece of context
from the local vector store
Confidential
10
©2023 Supermicro
Large Scale AI Training
• Key Technologies
• NVIDIA HGX H100 SXM 8-GPU/4-GPU with 900GB/s NVLink interconnect
• Dedicated, lots of high performance, high bandwidth GPU memory - HBM3, HBM2e
• 400GbE networking (Ethernet or InfiniBand), PCIe 5.0 storage for fast AI data pipe
• NVIDIA GPUDirect RDMA and Storage to keep feeding data to GPUs with minimum latency
• Liquid cooling for GPUs and CPUs
• All-flash storage and file systems to support petabytes of hot-tier data cache
• NVIDIA HGX H100 SXM5
board with 4- GPU or 8-
GPU
• NVLink and NVSwitch
• 80GB HBM3 per GPU
• Up to 700W TDP
• NVIDIA ConnectX-7
• Up to 400GbE or 400G NDR InfiniBand
• x16/x32 PCIe 5.0
Confidential
Supermicro AI
Experience
Supermicro AI
Experience
Marv Wexler
August 2023
Marv Wexler
August 2023
Better Faster Greener™ © 2023 Supermicro
Confidential
9/20/2023 Better Faster Greener™ © 2023 Supermicro
12
Confidential
Evolving to an AI / Total IT Solutions Partner
9/20/2023 Better Faster Greener™ © 2022 Supermicro
13
 5S: Software, Services,
Switch, Storage, Security
and more
 Total Solutions: Enterprise,
OEM- Appliance / Cloud
 Complete Systems
 Sub-systems and
Components
~5X+ Faster growth rate than the
industry avg rate over the past 12+
months (~50% YoY)
~5X+ Faster growth rate than the
industry avg rate over the past 12+
months (~50% YoY)
Our Momentum:
SMCI 1.0
Components &
Subsystems
SMCI 2.0
Servers &
Storage Systems
SMCI 3.0
Total IT
Solutions
Today
1993
$5B
$10B
Confidential
SMCI AI Strategy
9/20/2023 Better Faster Greener™ © 2023 Supermicro
14
• Partner with the Leaders
• Provide the best picks and shovels for the gold miners (Apps, YOU)
• Do not be religious with Products Offerings (multi-vendor, multi-platform)
Confidential
SMCI AI Business Results
9/20/2023 Better Faster Greener™ © 2023 Supermicro
15
• Bring up platform partner for virtually all AI Solutions / GPU offerings
• Lead supplier for virtually all Large Language Model Cloud Deployments
(ChatGPT, BARD, Bing, etc.)
The Next Platform, August 16, 2023
Confidential
16
©2023 Supermicro
GPU Optimized Systems by Workloads
• Large Scale AI Training • HPC/AI Workloads
H100 PCIe
Grace Hopper Superchip (Grace
CPU + H100 GPU)
H100 NVL
HGX H100 SXM
8-GPU or 4-GPU
4U 4-GPU System (HGX H100 SXM)
(codenamed: Redstone-Next)
SYS-421GU-TNXR, SYS-521GU-TNXR
8U 8-GPU System (HGX H100 SXM)
(codenamed: Delta-Next)
SYS-821GE-TNHR, AS -8125GS-TNHR
4U 4-GPU System (HGX H100 SXM)
SYS-421GU-TNXR
4U/5U 8-10 GPU System
SYS-521GE-TNRT, SYS-421GE-TNRT/TNRT3
AS -4125GS-TNRT/TNRT1/TNRT2
1U Grace Hopper MGX System
SYS-421GU-TNXR / SYS-521GU-TNXR
8U SuperBlade (Up to 20 nodes)
SBI-411E-1G / SBI-411E-5G
Petabyte Scale All-Flash Storage
SSG-121E-NE316R, ASG-1115S-NE316R
Confidential
Scales to thousands of nodes in 32-node increments
(SRS-42UHPC-32SU-01)
Accelerate AI Development by Supermicro
Supermicro 8U Delta-Next (SYS-821GE-TNHR)
A Proven Platform, Purpose Built for AI
H100 SXM5 GPU ConnectX-7 SmartNICs
H100 Rack Scale SuperPod Scalable Unit
8x NVIDIA H100 SXM5 GPUs | 640GB HBM3 GPU Memory 2TB
System Memory | 3.2Tbps Network B/W | Superior I/O
32x HGX H100 | 1+ EFLOPS AI | 20TB HBM3 GPU Memory 102.4Tbps
Network B/W Non-blocking | InfiniBand NDR
Software: NVIDIA BCM | NGC | NVAIE | SLURM | Kubernetes
Full Turnkey AI Supercomputer for Enterprises
9/20/2023 Better Faster Greener™ © 2023 Supermicro
17
Confidential
Supermicro Rack Integration Services
• Full rack integration up to L11 and L12
• Broad portfolio of compute, power, cooling
and networking options
• Liquid cooling integration
• Cooling Distribution Unit (CDU)
• Direct to Chip cold plate
• Manifold and tubing
• Design, assembly, configuration, testing
and deployment
• Start running applications from Day 1
Confidential
Supermicro CDU
80kW to 120kW, 45°C Warm Water
Liquid Cooling Option for Rack Scale H100 SuperPods
9/20/2023 Better Faster Greener™ © 2023 Supermicro
19
Confidential
Onsite Rack Services
9/20/2023 Better Faster Greener™ © 2023 Supermicro
20
Simplifying Your Solution Deployment Needs
• White glove custom service from beginning to end
• Onsite rack & stack of the custom solution
• Onsite integration ensuring proper installation and
connectivity, providing for reliable operation and reduced
downtime
• Onsite software installation with application configurations
• Onsite benchmark testing ensuring solution meets the
requirements of the customer
• Delivery of a customized rack solution that meets all
requirements
• SMC Cooling tower product line is available to enable
facility level water connections for CDU/CDM/RDHX
Reliable – Repeatable – Reproducible
Confidential
DISCLAIMER
Super Micro Computer, Inc. may make changes to specifications and product descriptions at any time, without notice. The
information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions
and typographical errors. Any performance tests and ratings are measured using systems that reflect the approximate
performance of Super Micro Computer, Inc. products as measured by those tests. Any differences in software or hardware
configuration may affect actual performance, and Super Micro Computer, Inc. does not control the design or implementation of
third party benchmarks or websites referenced in this document. The information contained herein is subject to change and may
be rendered inaccurate for many reasons, including but not limited to any changes in product and/or roadmap, component and
hardware revision changes, new model and/or product releases, software changes, firmware changes, or the like. Super Micro
Computer, Inc. assumes no obligation to update or otherwise correct or revise this information.
SUPER MICRO COMPUTER, INC. MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE
CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT
MAY APPEAR IN THIS INFORMATION.
SUPER MICRO COMPUTER, INC. SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR
FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL SUPER MICRO COMPUTER, INC. BE LIABLE TO ANY
PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF
ANY INFORMATION CONTAINED HEREIN, EVEN IF SUPER MICRO COMPUTER, Inc. IS EXPRESSLY ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
ATTRIBUTION
© 2023 Super Micro Computer, Inc. All rights reserved.
9/20/2023 Better Faster Greener™ © 2023 Supermicro
21
Confidential
www.supermicro.com
1 of 22

More Related Content

What's hot(20)

Matt Lewis - The Hardest Thing-Final to Host.pdfMatt Lewis - The Hardest Thing-Final to Host.pdf
Matt Lewis - The Hardest Thing-Final to Host.pdf
SOLTUIONSpeople, THINKubators, THINKathons•395 views
Nils Vesk - Building an Innovative, Productive, AI empowered Culture.pdfNils Vesk - Building an Innovative, Productive, AI empowered Culture.pdf
Nils Vesk - Building an Innovative, Productive, AI empowered Culture.pdf
SOLTUIONSpeople, THINKubators, THINKathons•404 views
 Neha Shukla - Future of the AI Revolution - Building Ethical and Equitable T... Neha Shukla - Future of the AI Revolution - Building Ethical and Equitable T...
Neha Shukla - Future of the AI Revolution - Building Ethical and Equitable T...
SOLTUIONSpeople, THINKubators, THINKathons•337 views
Tojin Eapen - Augmenting Creativity Using Gen AI.pdfTojin Eapen - Augmenting Creativity Using Gen AI.pdf
Tojin Eapen - Augmenting Creativity Using Gen AI.pdf
SOLTUIONSpeople, THINKubators, THINKathons•391 views
Carol Scott - Fast Track  Your AI Journey.pdfCarol Scott - Fast Track  Your AI Journey.pdf
Carol Scott - Fast Track Your AI Journey.pdf
SOLTUIONSpeople, THINKubators, THINKathons•453 views
Jordan Wilson - Expert Chats Train ChatGPT to be your employee with the PPP m...Jordan Wilson - Expert Chats Train ChatGPT to be your employee with the PPP m...
Jordan Wilson - Expert Chats Train ChatGPT to be your employee with the PPP m...
SOLTUIONSpeople, THINKubators, THINKathons•635 views
Ben Bressington - Buy Back Your Time and Increase Profits - 1 AI Strategy You...Ben Bressington - Buy Back Your Time and Increase Profits - 1 AI Strategy You...
Ben Bressington - Buy Back Your Time and Increase Profits - 1 AI Strategy You...
SOLTUIONSpeople, THINKubators, THINKathons•292 views
Ashen Bhatti - How I Build Companies with LLM.pdfAshen Bhatti - How I Build Companies with LLM.pdf
Ashen Bhatti - How I Build Companies with LLM.pdf
SOLTUIONSpeople, THINKubators, THINKathons•210 views
Generative AIGenerative AI
Generative AI
lutzsuarnaba1•614 views
Bryan Mattimore - AI Ideation and TIE.pdfBryan Mattimore - AI Ideation and TIE.pdf
Bryan Mattimore - AI Ideation and TIE.pdf
SOLTUIONSpeople, THINKubators, THINKathons•361 views
arbar https://www.slideshare.net/Solutionman/charles-caldwell-improve-your-li...arbar https://www.slideshare.net/Solutionman/charles-caldwell-improve-your-li...
arbar https://www.slideshare.net/Solutionman/charles-caldwell-improve-your-li...
SOLTUIONSpeople, THINKubators, THINKathons•306 views
Generative AI Risks & ConcernsGenerative AI Risks & Concerns
Generative AI Risks & Concerns
Ajitesh Kumar•2.5K views
Leveraging Generative AI & Best practicesLeveraging Generative AI & Best practices
Leveraging Generative AI & Best practices
DianaGray10•1.7K views
Tom Nodine - How AI Helps Us Live Longer.pdfTom Nodine - How AI Helps Us Live Longer.pdf
Tom Nodine - How AI Helps Us Live Longer.pdf
SOLTUIONSpeople, THINKubators, THINKathons•284 views
GENERATIVE AI, THE FUTURE OF PRODUCTIVITYGENERATIVE AI, THE FUTURE OF PRODUCTIVITY
GENERATIVE AI, THE FUTURE OF PRODUCTIVITY
Andre Muscat•6.5K views
The Future is in Responsible Generative AIThe Future is in Responsible Generative AI
The Future is in Responsible Generative AI
Saeed Al Dhaheri•657 views

Similar to Marv Wexler - Transform Your with AI.pdf(20)

Accelerating Innovation from Edge to CloudAccelerating Innovation from Edge to Cloud
Accelerating Innovation from Edge to Cloud
Rebekah Rodriguez•283 views
SUPERMICRO Innovative Computing ArchitectureSUPERMICRO Innovative Computing Architecture
SUPERMICRO Innovative Computing Architecture
Intel IT Center•1K views
Cimteq CableBuilder GoCimteq CableBuilder Go
Cimteq CableBuilder Go
Cimteq•143 views
Cisco connect montreal 2018   compute v finalCisco connect montreal 2018   compute v final
Cisco connect montreal 2018 compute v final
Cisco Canada•1.6K views
What is ThousandEyes WebinarWhat is ThousandEyes Webinar
What is ThousandEyes Webinar
ThousandEyes•61 views
abiquoabiquo
abiquo
guestf5c2fa•299 views
TechWiseTV Workshop: ASR 9000 TechWiseTV Workshop: ASR 9000
TechWiseTV Workshop: ASR 9000
Robb Boyd•734 views
InSource 2017 Roadshow: Analyzing DataInSource 2017 Roadshow: Analyzing Data
InSource 2017 Roadshow: Analyzing Data
InSource Solutions•1.3K views

More from SOLTUIONSpeople, THINKubators, THINKathons(12)

George Boretos & FutureUP-AI the big picture.pdfGeorge Boretos & FutureUP-AI the big picture.pdf
George Boretos & FutureUP-AI the big picture.pdf
SOLTUIONSpeople, THINKubators, THINKathons•385 views
Audrey Chia - Supercharge Your Growth.pdfAudrey Chia - Supercharge Your Growth.pdf
Audrey Chia - Supercharge Your Growth.pdf
SOLTUIONSpeople, THINKubators, THINKathons•315 views
Garima Gupta - How AI can Change your online learning experience.pdfGarima Gupta - How AI can Change your online learning experience.pdf
Garima Gupta - How AI can Change your online learning experience.pdf
SOLTUIONSpeople, THINKubators, THINKathons•244 views
Kai Wang - AI for Innovation1.1r.pdfKai Wang - AI for Innovation1.1r.pdf
Kai Wang - AI for Innovation1.1r.pdf
SOLTUIONSpeople, THINKubators, THINKathons•276 views
Lars Tvede - How We Boosted Total Productivity in Our Company by 20x w.pdfLars Tvede - How We Boosted Total Productivity in Our Company by 20x w.pdf
Lars Tvede - How We Boosted Total Productivity in Our Company by 20x w.pdf
SOLTUIONSpeople, THINKubators, THINKathons•225 views
Kelly Dowd - Leading Digital Transformation with AI and Human-Centered Design...Kelly Dowd - Leading Digital Transformation with AI and Human-Centered Design...
Kelly Dowd - Leading Digital Transformation with AI and Human-Centered Design...
SOLTUIONSpeople, THINKubators, THINKathons•200 views
Josh Cavalier - ChatGPT Prompt Strategies.pdfJosh Cavalier - ChatGPT Prompt Strategies.pdf
Josh Cavalier - ChatGPT Prompt Strategies.pdf
SOLTUIONSpeople, THINKubators, THINKathons•548 views
Jim Lecinski - Capturing the Power of AI in Marketing.pdfJim Lecinski - Capturing the Power of AI in Marketing.pdf
Jim Lecinski - Capturing the Power of AI in Marketing.pdf
SOLTUIONSpeople, THINKubators, THINKathons•575 views
Barbar Bahatti - Harnessing the Power of LLMs.pdfBarbar Bahatti - Harnessing the Power of LLMs.pdf
Barbar Bahatti - Harnessing the Power of LLMs.pdf
SOLTUIONSpeople, THINKubators, THINKathons•206 views
Charles Caldwell - Improve Your Life with AI.pdfCharles Caldwell - Improve Your Life with AI.pdf
Charles Caldwell - Improve Your Life with AI.pdf
SOLTUIONSpeople, THINKubators, THINKathons•390 views
Dr. Harvey Castro - GPT Healthcare.pdfDr. Harvey Castro - GPT Healthcare.pdf
Dr. Harvey Castro - GPT Healthcare.pdf
SOLTUIONSpeople, THINKubators, THINKathons•372 views
George Pace - Keeping Pace with ChatGPT.pdfGeorge Pace - Keeping Pace with ChatGPT.pdf
George Pace - Keeping Pace with ChatGPT.pdf
SOLTUIONSpeople, THINKubators, THINKathons•559 views

Recently uploaded(20)

terms_2.pdfterms_2.pdf
terms_2.pdf
JAWADIQBAL40•11 views
Effective Supervisory SkillEffective Supervisory Skill
Effective Supervisory Skill
Seta Wicaksana•13 views
ANTHROPOIDS WHITE PAPER.pdfANTHROPOIDS WHITE PAPER.pdf
ANTHROPOIDS WHITE PAPER.pdf
Anthropoids Nfts •34 views
Corporate DeckCorporate Deck
Corporate Deck
Equinox Gold Corp.•228 views
chung chi tam compact chiu axitchung chi tam compact chiu axit
chung chi tam compact chiu axit
MaiThiAnh•10 views
SESS Market TrendsSESS Market Trends
SESS Market Trends
Thorsten Zoerner•13 views
Car license plate holder.pdfCar license plate holder.pdf
Car license plate holder.pdf
JAWADIQBAL40•28 views
Concierge Services Business PlanConcierge Services Business Plan
Concierge Services Business Plan
Jessica Larson•11 views
202311017_UKRAINE_FACT_SHEET_PDA_51.pdf202311017_UKRAINE_FACT_SHEET_PDA_51.pdf
202311017_UKRAINE_FACT_SHEET_PDA_51.pdf
Rbc Rbcua•3.1K views
Forex secret Forex secret
Forex secret
konghatatih•10 views
Aircon Clinic Singapore Aircon Clinic Singapore
Aircon Clinic Singapore
manuaggarwal25•15 views

Marv Wexler - Transform Your with AI.pdf

  • 1. Confidential Transform Your Business With AI Transform Your Business With AI AI Summit Marv Wexler GM Technical Services September 21, 2023 AI Summit Marv Wexler GM Technical Services September 21, 2023 Better Faster Greener™ © 2023 Supermicro
  • 2. Confidential Where are we on the AI journey ? 9/20/2023 Better Faster Greener™ © 2023 Supermicro 2 “Once a new technology rolls over you, if you're not part of the steamroller, you're part of the road.” - Stewart Brand
  • 3. Confidential 9/20/2023 Better Faster Greener™ © 2023 Supermicro 3 Current AI Trends • Democratization of AI will continue • AI is a fundamental differentiator for businesses • Find deeper insights in data, real-time and at scale -Else your competitors surely will • Generative AI is becoming commercialized • AI ethics a top priority • Biased algorithms, Deep fakes, “Hallucinations” as a feature • Generative AI applications reign : Microsoft (Designer), Adobe (Firefly), Meta (Ad creation) • New regulations for safe and responsible practices • EU AI Act: Set of new rules that establish obligations for risks from artificial intelligence
  • 4. Confidential AI Applications 9/20/2023 Better Faster Greener™ © 2023 Supermicro 4 Deep Learning Solving complex problems Computer model taught to learn actions using images, texts and sounds Machine Learning Machines making decisions Building Machines with predictive algorithm and create predictive models Artificial Intelligence Simulate intelligence Building Smart Machines capable of performing intelligent tasks
  • 5. Confidential 9/20/2023 Better Faster Greener™ © 2023 Supermicro 5 Text Image Audio Video Games Text/ Voice prompt Generative AI models (also Large Language LLM, or Foundational Models) User Input What is Generative AI? Generative AI models are models that, when receiving a text prompt, give an output related to that input. The output can be text, image, audio, video, code etc. The ability for generative AI to produce useful, impressively synthesized text, images, and other types of content almost effortlessly based on a few text cues has already become an important business capability worthy of providing immense value to most knowledge workers
  • 6. Confidential The far-reaching impacts of Generative AI 9/20/2023 Better Faster Greener™ © 2023 Supermicro 6 Around 75% of the technology's value will be seen across four areas: • customer operations • marketing and sales • software engineering • research and development automating conversations with customers creating personalized messages for customers generating code generative design
  • 7. Confidential Customizable AI infrastructure for Generative AI 9/20/2023 Better Faster Greener™ © 2023 Supermicro 7 Training •compute intensive •massive datasets involved Fine-Tuning •Requires relatively less computational power Inferencing •Accelerators may be needed depending on type of application (batch/real-time) Various stages in building a Generative AI Application At Supermicro, We have you covered all the way with affordable, customizable and scalable solutions
  • 8. Confidential Application Details 9/20/2023 Better Faster Greener™ © 2023 Supermicro 8 LangChain Instructor Embeddings WizardLM / LLAMA • Ask questions to your documents AND learn from your documents using the power of LLMs. • 100% private, no data leaves your execution environment at any point. • You can ingest documents and ask questions without an internet connection! localGPT BUILT WITH • Text pre processed into chunks • Embedded in a vector space • Query search for similar chunks An instruction-finetuned text embedding model that can generate text embeddings tailored to any task by simply providing the task instruction, without any finetuning. Instructor achieves SOTA on 70 diverse embedding tasks! (e.g., classification, retrieval, clustering, text evaluation, etc.) and domains (e.g., science, finance, etc.) • WizardLM is a Llama variant trained with complex instructions • Evol-Instruct which leverages AI to "evolve" instructions
  • 9. Confidential Application Details 9/20/2023 Better Faster Greener™ © 2023 Supermicro 9 Ingest.py • uses LangChain tools to parse the document and create embeddings locally using Instructor Embeddings Chroma vector store • local vector database that stores the created embeddings Run_localGPT • uses local LLM to understand questions and create answers. Similarity Search • used to extract right piece of context from the local vector store
  • 10. Confidential 10 ©2023 Supermicro Large Scale AI Training • Key Technologies • NVIDIA HGX H100 SXM 8-GPU/4-GPU with 900GB/s NVLink interconnect • Dedicated, lots of high performance, high bandwidth GPU memory - HBM3, HBM2e • 400GbE networking (Ethernet or InfiniBand), PCIe 5.0 storage for fast AI data pipe • NVIDIA GPUDirect RDMA and Storage to keep feeding data to GPUs with minimum latency • Liquid cooling for GPUs and CPUs • All-flash storage and file systems to support petabytes of hot-tier data cache • NVIDIA HGX H100 SXM5 board with 4- GPU or 8- GPU • NVLink and NVSwitch • 80GB HBM3 per GPU • Up to 700W TDP • NVIDIA ConnectX-7 • Up to 400GbE or 400G NDR InfiniBand • x16/x32 PCIe 5.0
  • 11. Confidential Supermicro AI Experience Supermicro AI Experience Marv Wexler August 2023 Marv Wexler August 2023 Better Faster Greener™ © 2023 Supermicro
  • 12. Confidential 9/20/2023 Better Faster Greener™ © 2023 Supermicro 12
  • 13. Confidential Evolving to an AI / Total IT Solutions Partner 9/20/2023 Better Faster Greener™ © 2022 Supermicro 13  5S: Software, Services, Switch, Storage, Security and more  Total Solutions: Enterprise, OEM- Appliance / Cloud  Complete Systems  Sub-systems and Components ~5X+ Faster growth rate than the industry avg rate over the past 12+ months (~50% YoY) ~5X+ Faster growth rate than the industry avg rate over the past 12+ months (~50% YoY) Our Momentum: SMCI 1.0 Components & Subsystems SMCI 2.0 Servers & Storage Systems SMCI 3.0 Total IT Solutions Today 1993 $5B $10B
  • 14. Confidential SMCI AI Strategy 9/20/2023 Better Faster Greener™ © 2023 Supermicro 14 • Partner with the Leaders • Provide the best picks and shovels for the gold miners (Apps, YOU) • Do not be religious with Products Offerings (multi-vendor, multi-platform)
  • 15. Confidential SMCI AI Business Results 9/20/2023 Better Faster Greener™ © 2023 Supermicro 15 • Bring up platform partner for virtually all AI Solutions / GPU offerings • Lead supplier for virtually all Large Language Model Cloud Deployments (ChatGPT, BARD, Bing, etc.) The Next Platform, August 16, 2023
  • 16. Confidential 16 ©2023 Supermicro GPU Optimized Systems by Workloads • Large Scale AI Training • HPC/AI Workloads H100 PCIe Grace Hopper Superchip (Grace CPU + H100 GPU) H100 NVL HGX H100 SXM 8-GPU or 4-GPU 4U 4-GPU System (HGX H100 SXM) (codenamed: Redstone-Next) SYS-421GU-TNXR, SYS-521GU-TNXR 8U 8-GPU System (HGX H100 SXM) (codenamed: Delta-Next) SYS-821GE-TNHR, AS -8125GS-TNHR 4U 4-GPU System (HGX H100 SXM) SYS-421GU-TNXR 4U/5U 8-10 GPU System SYS-521GE-TNRT, SYS-421GE-TNRT/TNRT3 AS -4125GS-TNRT/TNRT1/TNRT2 1U Grace Hopper MGX System SYS-421GU-TNXR / SYS-521GU-TNXR 8U SuperBlade (Up to 20 nodes) SBI-411E-1G / SBI-411E-5G Petabyte Scale All-Flash Storage SSG-121E-NE316R, ASG-1115S-NE316R
  • 17. Confidential Scales to thousands of nodes in 32-node increments (SRS-42UHPC-32SU-01) Accelerate AI Development by Supermicro Supermicro 8U Delta-Next (SYS-821GE-TNHR) A Proven Platform, Purpose Built for AI H100 SXM5 GPU ConnectX-7 SmartNICs H100 Rack Scale SuperPod Scalable Unit 8x NVIDIA H100 SXM5 GPUs | 640GB HBM3 GPU Memory 2TB System Memory | 3.2Tbps Network B/W | Superior I/O 32x HGX H100 | 1+ EFLOPS AI | 20TB HBM3 GPU Memory 102.4Tbps Network B/W Non-blocking | InfiniBand NDR Software: NVIDIA BCM | NGC | NVAIE | SLURM | Kubernetes Full Turnkey AI Supercomputer for Enterprises 9/20/2023 Better Faster Greener™ © 2023 Supermicro 17
  • 18. Confidential Supermicro Rack Integration Services • Full rack integration up to L11 and L12 • Broad portfolio of compute, power, cooling and networking options • Liquid cooling integration • Cooling Distribution Unit (CDU) • Direct to Chip cold plate • Manifold and tubing • Design, assembly, configuration, testing and deployment • Start running applications from Day 1
  • 19. Confidential Supermicro CDU 80kW to 120kW, 45°C Warm Water Liquid Cooling Option for Rack Scale H100 SuperPods 9/20/2023 Better Faster Greener™ © 2023 Supermicro 19
  • 20. Confidential Onsite Rack Services 9/20/2023 Better Faster Greener™ © 2023 Supermicro 20 Simplifying Your Solution Deployment Needs • White glove custom service from beginning to end • Onsite rack & stack of the custom solution • Onsite integration ensuring proper installation and connectivity, providing for reliable operation and reduced downtime • Onsite software installation with application configurations • Onsite benchmark testing ensuring solution meets the requirements of the customer • Delivery of a customized rack solution that meets all requirements • SMC Cooling tower product line is available to enable facility level water connections for CDU/CDM/RDHX Reliable – Repeatable – Reproducible
  • 21. Confidential DISCLAIMER Super Micro Computer, Inc. may make changes to specifications and product descriptions at any time, without notice. The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. Any performance tests and ratings are measured using systems that reflect the approximate performance of Super Micro Computer, Inc. products as measured by those tests. Any differences in software or hardware configuration may affect actual performance, and Super Micro Computer, Inc. does not control the design or implementation of third party benchmarks or websites referenced in this document. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to any changes in product and/or roadmap, component and hardware revision changes, new model and/or product releases, software changes, firmware changes, or the like. Super Micro Computer, Inc. assumes no obligation to update or otherwise correct or revise this information. SUPER MICRO COMPUTER, INC. MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. SUPER MICRO COMPUTER, INC. SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL SUPER MICRO COMPUTER, INC. BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF SUPER MICRO COMPUTER, Inc. IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. ATTRIBUTION © 2023 Super Micro Computer, Inc. All rights reserved. 9/20/2023 Better Faster Greener™ © 2023 Supermicro 21