Chicago AWS Solutions Architect Mehdy Haghy recaps the new AI/ML releases and GenAI

1. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Generative AI & AI/ML on AWS
Re:invent re:cap 2023
Mehdy Haghy
Sr. Solutions Architect
mhaghy@amazon.com

2. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 2
Agenda
• Generative AI Stack
• Amazon Q! Your Expert.
• Amazon Bedrock
• Amazon Code Whisperer
• Amazon SageMaker
• Amazon AI Services – Gen AI Integrations
• Q&A

3. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 3
Generative AI Stack
Inferentia
Trainium SageMaker
Amazon Bedrock
Guardrails Agents Customization
Amazon Q
Amazon
QuickSight
Amazon
Connect
Amazon
CodeWhisperer
Knowledge Bases

4. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 4
Amazon Q
Y O U R G E N E R A T I V E A I A S S I S T A N T D E S I G N E D F O R W O R K
> Built to be secure and private
> Understands your company information,
code, and system
> Personalizes interactions based on
your role and permissions
> Engages in conversations to solve problems,
generate content, and take action

5. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 5
Amazon Q areas of expertise
Your
business
Amazon
QuickSight
Building
on AWS
Amazon
Connect
AWS
Supply Chain

6. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 6
Amazon Q is your business expert (Preview)
B O O S T Y O U R W O R K F O R C E P R O D U C T I V I T Y W I T H G E N E R A T I V E A I
6
> Delivers quick, accurate, and relevant answers
to your business questions, securely and
privately and document repositories.
> Provides responses with references and
citations for easy fact-checking
> Respects existing access control based on
user permissions
> Connects to over 40 popular enterprise
applications and document repositories
> Enables administrators to easily apply
guardrails to customize and control responses
Amazon Q
Amazon Q
Amazon Q

7. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 7
Amazon Q Builder

8. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 8
Amazon Q for QuickSight

9. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 9
Generative AI-powered agent assist delivers suggested
responses and actions
A M A Z O N Q I N C O N N E C T
Detected issue
Generated response
Generated solution
Articles and documents
used to generate the
response & solution

10. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 10
Amazon Q in AWS Supply Chain (coming soon)
10
> Using Amazon Q in AWS Supply
Chain, inventory managers, supply
and demand planners, and others will
be able to ask and get intelligent
answers about what is happening in
their supply chain, why it is
happening, and what actions to take.
> They will also be able to explore
what-if scenarios to understand the
tradeoffs between different supply
chain choices.

11. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 11
Amazon Bedrock
The easiest way to build and scale generative AI applications on LLMs and other FMs
Access a range of leading
FMs via a single API
Privately customize FMs
with your own data
Enable data security and compliance
Build agents that execute complex business
tasks by dynamically invoking APIs
Extend the power of FMs with your data using
retrieval augmented generation (RAG)
Get the best price performance
without managing infrastructure

12. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 12
New Models in Bedrock
Amazon Titan Image Generator (Preview)
Amazon Titan Multimodal Embeddings
Amazon Titan Text Lite and Express (now GA)
Stable Diffusion XL 1.0 (Now GA)
Added Llama2, Chat 13B and 70B
Claude 2.1 from Anthropic

13. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 13
Vector Databases for Amazon Bedrock and batch
inference
Vector Engine for Amazon OpenSearch
Redis Enterprise Cloud
Pinecone
Amazon Aurora
MongoDB
(Coming soon)
(Coming soon)
Amazon Bedrock now supports batch inference
Vector Search for Amazon DocumentDB

14. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 14
Custom Models in Amazon Bedrock (GA)
14
• Fine-tuning and continued pre-training
Deliver tailored,
differentiated tail
user experiences with
customized FMs
Fine-tune Llama 2,
Command, and Titan
FMs for specific tasks
with labeled data
Use continued pre-
training to adapt
Titan Text FMs to
your domain with
unlabeled data
None of your inputs
to or outputs from
Amazon Bedrock will
be used to train the
original base models
Meta—Llama2
Social media
Display ads
Web copy
GENERATED CONTENT
FINE-TUNED MODEL
AMAZON BEDROCK
Labeled data
AMAZON S3
Copy
Meta—Llama2

15. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 15
Evaluate, compare, and select the best FMs for your use case
Evaluate, compare, and select the best FMs for your use case in Amazon Bedrock (Preview)
Automatic evaluation and human evaluation using your own work team are available today in public preview in AWS Regions US East (N. Virginia) and US West
(Oregon). Human evaluation using an AWS managed team is available in public preview in AWS Region US East (N. Virginia).
Preview
Choose automatic or human
evaluation method
Curated datasets
or bring your own
Pre-defined and custom metrics
Human evaluation report
Automatic evaluation report

16. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 16
Guardrails for Amazon Bedrock
16
• Implement safeguards tailored to your application requirements and responsible AI policies
Preview
Apply guardrails consistently across FMs
including fine-tuned models and agents
Configure filtering of harmful content and
topics to avoid based on your responsible
AI policies
Redact personally identifiable
information (coming soon)
Guardrails for Amazon Bedrock is available today in limited preview, including Amazon Titan Text, Anthropic Claude, Meta Llama 2, AI21 Jurassic, and Cohere
Command. You can also use guardrails with custom models as well as Agents for Amazon Bedrock.

17. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 17
Agents and Fully managed RAG experience with Knowledge
Bases (GA)
Agents with improved control of orchestration and
visibility into reasoning
• Automatic prompt creation
• Retrieval augmented generation (RAG)
• Orchestrate and execute multistep tasks
• Trace through the Chain of Thought (CoT) reasoning
• Prompt engineering
Knowledge Bases now delivers fully managed RAG experience
• Fully managed support for end-to-end RAG workflow
• Securely connect FMs and agents to data sources
• Easily retrieve relevant data and augment prompts
• Provide source attribution
Agents & Knowledge bases for Amazon Bedrock are available today in AWS Regions US East (N. Virginia) and US West (Oregon).

18. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 18
PartyRock and Amazon Bedrock playground
Announcing PartyRock, an Amazon Bedrock Playground
New Discover Apps page for PartyRock, an Amazon Bedrock
Playground - https://partyrock.aws/discover

19. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 19
Amazon CodeWhisperer - Your AI-powered coding companion
customizations and further support
Support for Infrastructure as
Code (IaC), code remediation
and security scanning
Suggestions to remediate
security and code quality
issues that allows
development speeds up by
28%
now in Visual Studio 2022 &
Amazon CodeWhisperer for
command line (preview)

20. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 20
Amazon SageMaker
• Build, train, and deploy machine learning (ML) models at scale

21. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 21
Introducing Amazon SageMaker HyperPod
• Develop and train FMs continuously for weeks and months
• Reduce training time for large models up by 40%
Self-healing clusters reduce
training time by up to 20%
Resilient
environment
SageMaker distributed
training libraries improve
performance by up to 20%
Streamline
distributed training
Control over computing
environment and workload
scheduling
Optimized
resources utilization
21

22. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 22
3- Amazon SageMaker Studio
Run Jupyterlab 4 application within Studio
VS Code Support
New setup and onboarding experience
Includes latest SageMaker Distribution image
Bring your own EFS volume
Improved JumpStart Experience

23. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 23
Amazon SageMaker Inference, Training, MLOps & IDE
updates
•Inference Capabilities:
• Reduce model deployment costs by 50% on average.
• Achieve 20% lower inference latency on average.
• Deploy multiple models to the same instance for better resource utilization.
• Intelligent routing of inference requests based on available instances.
•Smart Data Sifting for Model Training (Preview):
• Automatically inspects and evaluates training data on-the-fly.
• Selectively learns from the most informative data samples.
• Reduces model training time and cost by up to 35%.
•Improved SDK Tooling and UX for Model Deployment:
• New Python SDK library simplifies packaging and deploying ML models.
• Reduces deployment process from seven steps to one.
• Option for local inference.
• New interactive UI experiences in SageMaker Studio for quick deployment.
•Large Model Inference (LMI) with TensorRT-LLM Support:
• Reduces latency by 33% on average.
• Improves throughput by 60% on average.
• Compatible with specific models like Llama2-70B, Falcon-40B, and
CodeLlama-34B.
•API Support for SageMaker Notebook Jobs:
• Programmatically run notebooks as jobs using SageMaker Pipelines.
• Integrated with SageMaker's ML workflow orchestration service.
•Low Code Data Preparation with SageMaker Data Wrangler:
• Launchable from EMR Studio.
• Visual interface with 300+ transformations backed by Spark.
• Analyze, clean, and create ML features without changes to existing pipelines.
•Geospatial Processing Jobs:
• Standardized geospatial container for accessing data catalog.
• Process data with open-source algorithms or pre-trained ML models.
• Visualize predictions on a map and collaborate with team members.
•ML Feature Pipelines with SageMaker Feature Store:
• Connect to streaming data sources and data warehouses.
• Author transforms with Spark Structured Streaming.
• Schedule or trigger feature processing using Amazon EventBridge rules.
•Instance Expansions:
• Introduction of ml.p5.48xlarge instances in the US.
• Regional expansion of ml.p4d, ml.trn1, and ml.g5 instances (e.g., Tokyo, Seoul,
Singapore) for SageMaker Inference.
•Amazon Redshift ML (Preview) - Large Language Model Support:
• Leverage pretrained LLMs in SageMaker JumpStart within Redshift ML.
• Perform inferences on product feedback data for tasks like summarization, entity
extraction, sentiment analysis, and classification.
•New Setup and Onboarding Experience:
• Individual users can create a SageMaker domain with default pre-sets.
• Customizable features like Code Editor access in SageMaker Studio.

24. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 24
SageMaker Clarify for FM evaluations
Amazon SageMaker Clarify now supports
foundation model (FM) evaluations in preview
You can compare, and select FMs based on metrics
such as accuracy, robustness, bias, and toxicity, in
minutes.
This capability is available in select regions in
preview: US East (N. Virginia), US East (Ohio), US
West (Oregon), Asia Pacific (Tokyo), Asia Pacific
(Singapore), Europe (Frankfurt), Europe (Ireland).
For additional details, see our documentation and
pricing page.

25. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 25
SageMaker Canvas Updates - Do more with no-code
(with FMs & BI)
1- Leverage FMs for business analysis at scale
Foundation models now available in Amazon SageMaker
Canvas
2- Natural language instructions for data preparation,
content summarization and information extraction
(integration with Kendra and Bedrock/Jumpstarts)
3- Advanced configurations & Model Leaderboard &
Deploy ML models built in SageMaker Canvas to
SageMaker real-time endpoints
4- Amazon QuickSight announces predictive analytics
using Amazon SageMaker Canvas - With this new
capability, you can evolve your analytics from descriptive to
predictive capabilities, enabling the entire organization with a
forward-looking view of the business.

26. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 26
AI Services

27. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 27
AWS HealthScribe
27
A purpose-built HIPAA-eligible, generative AI service
empowering healthcare software vendors to build clinical
applications that automatically transcribe and summarize
patient-clinician conversations.
Now
GA

28. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 28
AWS AI Service Cards
Amazon Comprehend Detect PII
Amazon Transcribe Toxicity Detection
Amazon Rekognition Face Liveness
New Service
Cards
TRANSPARENCY RESOURCE TO ADVANCE RESPONSIBLE AI
AWS HealthScribe

29. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 29
Amazon Personalize
Themes for recommendations
Next-Best-Action recipe
ACCELERATE YOUR BUSINESS GROWTH WITH PERSONALIZED USER EXPERIENCES
Recommend the right action to the right user
proactively
Gen AI based Content generator identifies
thematic connections between recommended
items
OpenSearch integration

30. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 30
Amazon Lex
Assisted Slot Resolution with Gen AI
Conversational FAQ with Gen AI (Preview)
CONVERSATIONAL AI FOR VOICE AND TEXT INTERFACES
Descriptive Bot Builder using Gen AI
Generate utterances using Gen AI

31. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 31
Amazon Transcribe
Automatic Speech Recognition for 100+ languages
Automatic Language Identification for Real-Time Streams
Call Analytics - Generative AI Call Summaries
UNLOCKING VALUE WITH SPEECH TO TEXT

32. Thank you!
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Web Services, AWS, the Powered by AWS logo, and all AWS service
names used in this slide deck are trademarks of Amazon.com, Inc. or its affiliates.
Mehdy Haghy
Sr. Solutions Architect
mhaghy@amazon.com