Presentation on generative artificial intelligence (GenAI) given to UC Davis IT leadership November 1st, 2023.

1. GenAI
Adam Getchell (acgetchell@ucdavis.edu)
Topic list generated by ChatGPT 4.0
Slides designed with the help of Microsoft Office intelligent services

2. Some Implications for IT Management
Strategic Integration
• Provisioning GenAI capable HPC for researchers
• Teaching end-users how to efficiently use GenAI
• GenAI integration into existing services
• Training your own
Ethical Considerations
Future Trends
• GenAI in cybercrime
• Security tools
• Industry impact

3. Strategic
Integration
Q U O T E
Date Quote #
08/17/23 AD3Q6957
Quote To: Joe Lipman
UC Davis Sr. Sales Engineer
Adam Getchell Phone: 858-716-8258
Fax: 858-716-8233
E-mail: joe.lipman@advancedhpc.com
Phone:
Terms FOB
Net 30 Dest PPA
Thank you for the opportunity to provide the following proposal.
Qty Description Part Number Unit Price Ext. Price
1 $70,643.67 $70,643.67
AH-GPU204-SA01
Mercury GPU204 2U Server
AH-GPU204-SA01
Mercury GPU204 2U Server Includes:
Two (2) EPYC 7543 2.8 GHz Thirty-Two-Core 225W Processor
1024GB DDR4 3200MHz Memory (16 x 64GB Sticks)
Four (4) A100 80GB GPUs NVLINK
Four (4) Nvidia EDU Rebate Included (4x -$2,625)
One (1) 240GB SSD (OS)
One (1) 15.36TB NVMe 1DWPD (Scratch)
One (1) Quad Port 10G SFP+ Adapter
One (1) ConnectX®-6 VPI Adapter Card, 100Gb/s (HDR100, EDR IB
and 100GbE), Single-Port QSFP56 PCIe3.0/4.0 x16
One (1) SFT-DCMS-SINGLE
High Density 2U System with NVIDIA® HGX A100 4-GPU
Supports Four A100 80GB SXM4 GPUs
Direct Connect PCI-E Gen4 Platform with NVIDIA® NVLink
On Board BMC Supports Integrated IPMI 2.0 + KVM with Dedicated
10G LAN
Dual AMD EPYC 7002 Series Processors
8TB Registered ECC DDR4 3200MHz SDRAM in Thirty-Two DIMMs
Four PCI-E Gen 4 x16 (LP), One PCI-E Gen 4 x8 (LP)
Four Hot-Swap 2.5 Inch Drive Bays (SAS/SATA/NVMe Hybrid)
Two 2200W Redundant Power Supplies, Titanium Level + Four
Hot-Swap Heavy Duty Fans
Five Year Silver Plus Next Business Day Warranty on Parts and
Labor:
Toll-Free Phone/Email Support Help Desk Available Monday through
Friday, 9am-5pm PST.
Next Business Day Advance Replacement of All User Replaceable
Parts: Disk Drives, Cooling Fan Modules, Power Supply Modules,
System Memory and Software.
For System and All Other Components, System must be returned to
Advanced HPC.
Prepaid Return Shipping.
Free System Firmware Updates.
Lifetime Technical Support.
NOTE: This warranty does not include packaging.
System must be shipped in manufacturer packaging and palletized
at customer expense.
PLEASE KEEP THE MANUFACTURER PACKAGING FOR YOUR
SYSTEM.
Advanced HPC, Inc.
Phone: (858) 716-8262
Corporate Headquarters
Fax: (858) 716-8233
8228 Mercury Court, Suite 100
Page 1
San Diego CA 92111-1232 URL: www.advancedhpc.com
1 of 2

4. GenAI on HPC
Overwhelming demand for GPUs
Farm GPU node:
• 2 Epyc 7543(32-core)CPU
• 1024GB DDR4 3200MHz
• 4 A10080GBNVLink
• ConnectX-6 HDR100EDR Infiniband
~$70K, expecteddelivery 2024Q1

5. A100 80GB NVLink
4 GPUs interconnected @ 600 GB/s
• Per GPU
Cores:
• FP32: 6,912
• FP64: 3,456
• Tensor: 422
Clock: 1.065GHz
Default TDP: 400W
FLOPS:
• FP32: 19.5 TeraFLOPS*
• FP64: 19.5 TeraFLOPS
• Tensor:
• TF32: 312 TeraFLOPS*
• TF16: 624 TeraFLOPS*
• TF8: 1248 TeraFLOPS*
Price: ~$13K
*Sparse workloads with GEMM
Vendor claims 250X perf over CPUs

12. Training your
own
• Customization
• Reduced dependencies
• Cost efficiency
https://blog.replit.com/llm-
training

13. Ethical
Considerations
Taken from Figure 3 of Naveed, Humza, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed
Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian.“A Comprehensive
Overview of Large Language Models.”arXiv, October 5, 2023.
https://doi.org/10.48550/arXiv.2307.06435.

14. AGI mis-Alignment
Image by Jlleon, 9 April 2023, https://commons.wikimedia.org/wiki/File:Power-Seeking_Image.png

15. Will AGI Emerge from LLMs?
Probably not.
https://yuxili.substack.com/p/will-agi-emerge-from-large-language

16. HIPAA and
Privacy
Azure OpenAI Service
Prompts
• Not available to other customers
• Not available to OpenAI
• Not used to improve OpenAI
• Not used to improve any Microsoft or
3rd party products
• Fine-tuned Azure OpenAI models are
exclusive to UC Davis
https://learn.microsoft.com/en-
us/legal/cognitive-services/openai/data-
privacy

17. Executive Order on Safe, Secure, and
Trustworthy Artificial Intelligence
Share safety test results
with US government
Develop standards,
tools, and tests
Protect against risks of
using AI to engineer
dangerous biological
materials
Protect Americans from
AI-enabled fraud … by
establishing standards …
for detecting AI-
generated content
Establish an advanced
cybersecurity program to
develop AI tools to find
and fix vulnerabilities in
critical software
Order … a National
Security Memorandum
that directs further
actions on AI and
security

18. Future
Trends

19. GenAI in
Cybercrime
• Cybercriminals stole $6.9B in
2021 using Social Engineering
• Emulating your brand voice
• Spear-phishing
• Fake social media
• AI impersonationof VIPs

21. Which industries will AI have a big impact on?
EDUCATION HEALTHCARE FINANCE LAW TRANSPORTATION

22. References
1. https://chat.openai.com
2. https://support.microsoft.com/en-us/office/create-professional-slide-layouts-with-designer-53c77d7b-dc40-45c2-b684-81415eac0617
3. https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/a100-80gb-datasheet-update-nvidia-us-1521051-r2-web.pdf
4. https://www.nvidia.com/en-us/data-center/nvlink/
5. https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html
6. https://datalab.ucdavis.edu/2023/04/04/course-announcement-nlp-and-large-language-models-for-health-and-medicine/
7. https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/
8. https://aws.amazon.com/ai/
9. https://azure.microsoft.com/en-us/products/ai-services?activetab=pivot:azureopenaiservicetab
10. https://news.zoom.us/zoom-ai-companion/
11. https://blog.replit.com/llm-training
12. https://arxiv.org/pdf/2307.06435.pdf
13. https://yuxili.substack.com/p/will-agi-emerge-from-large-language
14. https://www.whitehouse.gov/briefing-room/statements-releases/2023/10/30/fact-sheet-president-biden-issues-executive-order-on-safe-secure-and-trustworthy-artificial-
intelligence/
15. https://www.imperva.com/learn/application-security/social-engineering-attack/
16. https://www.forbes.com/sites/zacharysmith/2022/03/22/cybercriminals-stole-69-billion-in-2021-using-social-engineering-to-break-into-remote-workplaces/
17. https://www.cloudwards.net/cyber-security-statistics/
18. https://bitwarden.com/rachel-tobac-ebook/
19. https://www.microsoft.com/en-us/security/business/ai-machine-learning/microsoft-security-copilot
20. https://www.techtarget.com/searchEnterpriseAI/tip/The-future-of-AI-What-to-expect-in-the-next-5-years