SlideShare a Scribd company logo
1 of 32
Download to read offline
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Generative AI & AI/ML on AWS
Re:invent re:cap 2023
Mehdy Haghy
Sr. Solutions Architect
mhaghy@amazon.com
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 2
Agenda
• Generative AI Stack
• Amazon Q! Your Expert.
• Amazon Bedrock
• Amazon Code Whisperer
• Amazon SageMaker
• Amazon AI Services – Gen AI Integrations
• Q&A
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 3
Generative AI Stack
Inferentia
Trainium SageMaker
Amazon Bedrock
Guardrails Agents Customization
Amazon Q
Amazon
QuickSight
Amazon
Connect
Amazon
CodeWhisperer
Knowledge Bases
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 4
Amazon Q
Y O U R G E N E R A T I V E A I A S S I S T A N T D E S I G N E D F O R W O R K
> Built to be secure and private
> Understands your company information,
code, and system
> Personalizes interactions based on
your role and permissions
> Engages in conversations to solve problems,
generate content, and take action
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 5
Amazon Q areas of expertise
Your
business
Amazon
QuickSight
Building
on AWS
Amazon
Connect
AWS
Supply Chain
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 6
Amazon Q is your business expert (Preview)
B O O S T Y O U R W O R K F O R C E P R O D U C T I V I T Y W I T H G E N E R A T I V E A I
6
> Delivers quick, accurate, and relevant answers
to your business questions, securely and
privately and document repositories.
> Provides responses with references and
citations for easy fact-checking
> Respects existing access control based on
user permissions
> Connects to over 40 popular enterprise
applications and document repositories
> Enables administrators to easily apply
guardrails to customize and control responses
Amazon Q
Amazon Q
Amazon Q
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 7
Amazon Q Builder
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 8
Amazon Q for QuickSight
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 9
Generative AI-powered agent assist delivers suggested
responses and actions
A M A Z O N Q I N C O N N E C T
Detected issue
Generated response
Generated solution
Articles and documents
used to generate the
response & solution
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 10
Amazon Q in AWS Supply Chain (coming soon)
10
> Using Amazon Q in AWS Supply
Chain, inventory managers, supply
and demand planners, and others will
be able to ask and get intelligent
answers about what is happening in
their supply chain, why it is
happening, and what actions to take.
> They will also be able to explore
what-if scenarios to understand the
tradeoffs between different supply
chain choices.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 11
Amazon Bedrock
The easiest way to build and scale generative AI applications on LLMs and other FMs
Access a range of leading
FMs via a single API
Privately customize FMs
with your own data
Enable data security and compliance
Build agents that execute complex business
tasks by dynamically invoking APIs
Extend the power of FMs with your data using
retrieval augmented generation (RAG)
Get the best price performance
without managing infrastructure
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 12
New Models in Bedrock
Amazon Titan Image Generator (Preview)
Amazon Titan Multimodal Embeddings
Amazon Titan Text Lite and Express (now GA)
Stable Diffusion XL 1.0 (Now GA)
Added Llama2, Chat 13B and 70B
Claude 2.1 from Anthropic
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 13
Vector Databases for Amazon Bedrock and batch
inference
Vector Engine for Amazon OpenSearch
Redis Enterprise Cloud
Pinecone
Amazon Aurora
MongoDB
(Coming soon)
(Coming soon)
Amazon Bedrock now supports batch inference
Vector Search for Amazon DocumentDB
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 14
Custom Models in Amazon Bedrock (GA)
14
• Fine-tuning and continued pre-training
Deliver tailored,
differentiated tail
user experiences with
customized FMs
Fine-tune Llama 2,
Command, and Titan
FMs for specific tasks
with labeled data
Use continued pre-
training to adapt
Titan Text FMs to
your domain with
unlabeled data
None of your inputs
to or outputs from
Amazon Bedrock will
be used to train the
original base models
Meta—Llama2
Social media
Display ads
Web copy
GENERATED CONTENT
FINE-TUNED MODEL
AMAZON BEDROCK
Labeled data
AMAZON S3
Copy
Meta—Llama2
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 15
Evaluate, compare, and select the best FMs for your use case
Evaluate, compare, and select the best FMs for your use case in Amazon Bedrock (Preview)
Automatic evaluation and human evaluation using your own work team are available today in public preview in AWS Regions US East (N. Virginia) and US West
(Oregon). Human evaluation using an AWS managed team is available in public preview in AWS Region US East (N. Virginia).
Preview
Choose automatic or human
evaluation method
Curated datasets
or bring your own
Pre-defined and custom metrics
Human evaluation report
Automatic evaluation report
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 16
Guardrails for Amazon Bedrock
16
• Implement safeguards tailored to your application requirements and responsible AI policies
Preview
Apply guardrails consistently across FMs
including fine-tuned models and agents
Configure filtering of harmful content and
topics to avoid based on your responsible
AI policies
Redact personally identifiable
information (coming soon)
Guardrails for Amazon Bedrock is available today in limited preview, including Amazon Titan Text, Anthropic Claude, Meta Llama 2, AI21 Jurassic, and Cohere
Command. You can also use guardrails with custom models as well as Agents for Amazon Bedrock.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 17
Agents and Fully managed RAG experience with Knowledge
Bases (GA)
Agents with improved control of orchestration and
visibility into reasoning
• Automatic prompt creation
• Retrieval augmented generation (RAG)
• Orchestrate and execute multistep tasks
• Trace through the Chain of Thought (CoT) reasoning
• Prompt engineering
Knowledge Bases now delivers fully managed RAG experience
• Fully managed support for end-to-end RAG workflow
• Securely connect FMs and agents to data sources
• Easily retrieve relevant data and augment prompts
• Provide source attribution
Agents & Knowledge bases for Amazon Bedrock are available today in AWS Regions US East (N. Virginia) and US West (Oregon).
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 18
PartyRock and Amazon Bedrock playground
Announcing PartyRock, an Amazon Bedrock Playground
New Discover Apps page for PartyRock, an Amazon Bedrock
Playground - https://partyrock.aws/discover
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 19
Amazon CodeWhisperer - Your AI-powered coding companion
customizations and further support
Support for Infrastructure as
Code (IaC), code remediation
and security scanning
Suggestions to remediate
security and code quality
issues that allows
development speeds up by
28%
now in Visual Studio 2022 &
Amazon CodeWhisperer for
command line (preview)
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 20
Amazon SageMaker
• Build, train, and deploy machine learning (ML) models at scale
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 21
Introducing Amazon SageMaker HyperPod
• Develop and train FMs continuously for weeks and months
• Reduce training time for large models up by 40%
Self-healing clusters reduce
training time by up to 20%
Resilient
environment
SageMaker distributed
training libraries improve
performance by up to 20%
Streamline
distributed training
Control over computing
environment and workload
scheduling
Optimized
resources utilization
21
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 22
3- Amazon SageMaker Studio
Run Jupyterlab 4 application within Studio
VS Code Support
New setup and onboarding experience
Includes latest SageMaker Distribution image
Bring your own EFS volume
Improved JumpStart Experience
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 23
Amazon SageMaker Inference, Training, MLOps & IDE
updates
•Inference Capabilities:
• Reduce model deployment costs by 50% on average.
• Achieve 20% lower inference latency on average.
• Deploy multiple models to the same instance for better resource utilization.
• Intelligent routing of inference requests based on available instances.
•Smart Data Sifting for Model Training (Preview):
• Automatically inspects and evaluates training data on-the-fly.
• Selectively learns from the most informative data samples.
• Reduces model training time and cost by up to 35%.
•Improved SDK Tooling and UX for Model Deployment:
• New Python SDK library simplifies packaging and deploying ML models.
• Reduces deployment process from seven steps to one.
• Option for local inference.
• New interactive UI experiences in SageMaker Studio for quick deployment.
•Large Model Inference (LMI) with TensorRT-LLM Support:
• Reduces latency by 33% on average.
• Improves throughput by 60% on average.
• Compatible with specific models like Llama2-70B, Falcon-40B, and
CodeLlama-34B.
•API Support for SageMaker Notebook Jobs:
• Programmatically run notebooks as jobs using SageMaker Pipelines.
• Integrated with SageMaker's ML workflow orchestration service.
•Low Code Data Preparation with SageMaker Data Wrangler:
• Launchable from EMR Studio.
• Visual interface with 300+ transformations backed by Spark.
• Analyze, clean, and create ML features without changes to existing pipelines.
•Geospatial Processing Jobs:
• Standardized geospatial container for accessing data catalog.
• Process data with open-source algorithms or pre-trained ML models.
• Visualize predictions on a map and collaborate with team members.
•ML Feature Pipelines with SageMaker Feature Store:
• Connect to streaming data sources and data warehouses.
• Author transforms with Spark Structured Streaming.
• Schedule or trigger feature processing using Amazon EventBridge rules.
•Instance Expansions:
• Introduction of ml.p5.48xlarge instances in the US.
• Regional expansion of ml.p4d, ml.trn1, and ml.g5 instances (e.g., Tokyo, Seoul,
Singapore) for SageMaker Inference.
•Amazon Redshift ML (Preview) - Large Language Model Support:
• Leverage pretrained LLMs in SageMaker JumpStart within Redshift ML.
• Perform inferences on product feedback data for tasks like summarization, entity
extraction, sentiment analysis, and classification.
•New Setup and Onboarding Experience:
• Individual users can create a SageMaker domain with default pre-sets.
• Customizable features like Code Editor access in SageMaker Studio.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 24
SageMaker Clarify for FM evaluations
Amazon SageMaker Clarify now supports
foundation model (FM) evaluations in preview
You can compare, and select FMs based on metrics
such as accuracy, robustness, bias, and toxicity, in
minutes.
This capability is available in select regions in
preview: US East (N. Virginia), US East (Ohio), US
West (Oregon), Asia Pacific (Tokyo), Asia Pacific
(Singapore), Europe (Frankfurt), Europe (Ireland).
For additional details, see our documentation and
pricing page.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 25
SageMaker Canvas Updates - Do more with no-code
(with FMs & BI)
1- Leverage FMs for business analysis at scale
Foundation models now available in Amazon SageMaker
Canvas
2- Natural language instructions for data preparation,
content summarization and information extraction
(integration with Kendra and Bedrock/Jumpstarts)
3- Advanced configurations & Model Leaderboard &
Deploy ML models built in SageMaker Canvas to
SageMaker real-time endpoints
4- Amazon QuickSight announces predictive analytics
using Amazon SageMaker Canvas - With this new
capability, you can evolve your analytics from descriptive to
predictive capabilities, enabling the entire organization with a
forward-looking view of the business.
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 26
AI Services
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 27
AWS HealthScribe
27
A purpose-built HIPAA-eligible, generative AI service
empowering healthcare software vendors to build clinical
applications that automatically transcribe and summarize
patient-clinician conversations.
Now
GA
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 28
AWS AI Service Cards
Amazon Comprehend Detect PII
Amazon Transcribe Toxicity Detection
Amazon Rekognition Face Liveness
New Service
Cards
TRANSPARENCY RESOURCE TO ADVANCE RESPONSIBLE AI
AWS HealthScribe
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 29
Amazon Personalize
Themes for recommendations
Next-Best-Action recipe
ACCELERATE YOUR BUSINESS GROWTH WITH PERSONALIZED USER EXPERIENCES
Recommend the right action to the right user
proactively
Gen AI based Content generator identifies
thematic connections between recommended
items
OpenSearch integration
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 30
Amazon Lex
Assisted Slot Resolution with Gen AI
Conversational FAQ with Gen AI (Preview)
CONVERSATIONAL AI FOR VOICE AND TEXT INTERFACES
Descriptive Bot Builder using Gen AI
Generate utterances using Gen AI
© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 31
Amazon Transcribe
Automatic Speech Recognition for 100+ languages
Automatic Language Identification for Real-Time Streams
Call Analytics - Generative AI Call Summaries
UNLOCKING VALUE WITH SPEECH TO TEXT
Thank you!
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Web Services, AWS, the Powered by AWS logo, and all AWS service
names used in this slide deck are trademarks of Amazon.com, Inc. or its affiliates.
Mehdy Haghy
Sr. Solutions Architect
mhaghy@amazon.com

More Related Content

Similar to Chicago AWS Solutions Architect Mehdy Haghy recaps the new AI/ML releases and GenAI 2023 AWS reInvent GenAI AIML recap

Similar to Chicago AWS Solutions Architect Mehdy Haghy recaps the new AI/ML releases and GenAI 2023 AWS reInvent GenAI AIML recap (20)

WhereML a Serverless ML Powered Location Guessing Twitter Bot
WhereML a Serverless ML Powered Location Guessing Twitter BotWhereML a Serverless ML Powered Location Guessing Twitter Bot
WhereML a Serverless ML Powered Location Guessing Twitter Bot
 
Suresh Poopandi_Generative AI On AWS-MidWestCommunityDay-Final.pdf
Suresh Poopandi_Generative AI On AWS-MidWestCommunityDay-Final.pdfSuresh Poopandi_Generative AI On AWS-MidWestCommunityDay-Final.pdf
Suresh Poopandi_Generative AI On AWS-MidWestCommunityDay-Final.pdf
 
Perform Machine Learning at the IoT Edge using AWS Greengrass and Amazon Sage...
Perform Machine Learning at the IoT Edge using AWS Greengrass and Amazon Sage...Perform Machine Learning at the IoT Edge using AWS Greengrass and Amazon Sage...
Perform Machine Learning at the IoT Edge using AWS Greengrass and Amazon Sage...
 
Module 3 - QuickSight Overview
Module 3 - QuickSight OverviewModule 3 - QuickSight Overview
Module 3 - QuickSight Overview
 
AWS Data Analytics on AWS
AWS Data Analytics on AWSAWS Data Analytics on AWS
AWS Data Analytics on AWS
 
Re-Invent 23 recap @ AWS UserGroup meetup
Re-Invent 23 recap @ AWS UserGroup meetupRe-Invent 23 recap @ AWS UserGroup meetup
Re-Invent 23 recap @ AWS UserGroup meetup
 
Build, Train, and Deploy ML Models Quickly and Easily with Amazon SageMaker, ...
Build, Train, and Deploy ML Models Quickly and Easily with Amazon SageMaker, ...Build, Train, and Deploy ML Models Quickly and Easily with Amazon SageMaker, ...
Build, Train, and Deploy ML Models Quickly and Easily with Amazon SageMaker, ...
 
Ensuring Your Windows Server Workloads Are Well-Architected - AWS Online Tech...
Ensuring Your Windows Server Workloads Are Well-Architected - AWS Online Tech...Ensuring Your Windows Server Workloads Are Well-Architected - AWS Online Tech...
Ensuring Your Windows Server Workloads Are Well-Architected - AWS Online Tech...
 
AI ﹑大數據媒體應用和利用機器學習與 AWS 媒體服務實現自動化內容生成
AI ﹑大數據媒體應用和利用機器學習與 AWS 媒體服務實現自動化內容生成AI ﹑大數據媒體應用和利用機器學習與 AWS 媒體服務實現自動化內容生成
AI ﹑大數據媒體應用和利用機器學習與 AWS 媒體服務實現自動化內容生成
 
Operational Excellence for Identity & Access Management (SEC334) - AWS re:Inv...
Operational Excellence for Identity & Access Management (SEC334) - AWS re:Inv...Operational Excellence for Identity & Access Management (SEC334) - AWS re:Inv...
Operational Excellence for Identity & Access Management (SEC334) - AWS re:Inv...
 
Building a Serverless AI Powered Twitter Bot: Collision 2018
Building a Serverless AI Powered Twitter Bot: Collision 2018Building a Serverless AI Powered Twitter Bot: Collision 2018
Building a Serverless AI Powered Twitter Bot: Collision 2018
 
Where ml ai_heavy
Where ml ai_heavyWhere ml ai_heavy
Where ml ai_heavy
 
Göteborg Reinvent 2023_Aritra_updated.pptx
Göteborg Reinvent 2023_Aritra_updated.pptxGöteborg Reinvent 2023_Aritra_updated.pptx
Göteborg Reinvent 2023_Aritra_updated.pptx
 
Single View of Data
Single View of DataSingle View of Data
Single View of Data
 
Data Security in the Cloud - Matt Taylor - AWS TechShift ANZ 2018
Data Security in the Cloud - Matt Taylor - AWS TechShift ANZ 2018Data Security in the Cloud - Matt Taylor - AWS TechShift ANZ 2018
Data Security in the Cloud - Matt Taylor - AWS TechShift ANZ 2018
 
Architect Your Legacy Microsoft Apps into Modern Cloud Workloads
 Architect Your Legacy Microsoft Apps into Modern Cloud Workloads Architect Your Legacy Microsoft Apps into Modern Cloud Workloads
Architect Your Legacy Microsoft Apps into Modern Cloud Workloads
 
BDA309 Build Your First Big Data Application on AWS
BDA309 Build Your First Big Data Application on AWSBDA309 Build Your First Big Data Application on AWS
BDA309 Build Your First Big Data Application on AWS
 
Building WhereML, an AI Powered Twitter Bot for Guessing Locations of Picture...
Building WhereML, an AI Powered Twitter Bot for Guessing Locations of Picture...Building WhereML, an AI Powered Twitter Bot for Guessing Locations of Picture...
Building WhereML, an AI Powered Twitter Bot for Guessing Locations of Picture...
 
Nuvem Híbrida - EBC on the road Brazil Edition [Portuguese]
Nuvem Híbrida - EBC on the road Brazil Edition [Portuguese]Nuvem Híbrida - EBC on the road Brazil Edition [Portuguese]
Nuvem Híbrida - EBC on the road Brazil Edition [Portuguese]
 
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
Analyzing your web and application logs with Cloudfront and ElasticSearch Ser...
 

More from AWS Chicago

More from AWS Chicago (20)

AWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user group
 
WilliamCollins_Road-to-Transit-Gateway.pptx
WilliamCollins_Road-to-Transit-Gateway.pptxWilliamCollins_Road-to-Transit-Gateway.pptx
WilliamCollins_Road-to-Transit-Gateway.pptx
 
Streamlined Entitlements with AWS Lake Formation - Anusha Dwivedula
Streamlined Entitlements with AWS Lake Formation - Anusha DwivedulaStreamlined Entitlements with AWS Lake Formation - Anusha Dwivedula
Streamlined Entitlements with AWS Lake Formation - Anusha Dwivedula
 
Steve Seaney_AWS Control Tower - 2023 Midwest Community Day - Final.pptx
Steve Seaney_AWS Control Tower - 2023 Midwest Community Day - Final.pptxSteve Seaney_AWS Control Tower - 2023 Midwest Community Day - Final.pptx
Steve Seaney_AWS Control Tower - 2023 Midwest Community Day - Final.pptx
 
Saurabh_Shanbhag - Building_SaaS_on_AWS.pptx
Saurabh_Shanbhag - Building_SaaS_on_AWS.pptxSaurabh_Shanbhag - Building_SaaS_on_AWS.pptx
Saurabh_Shanbhag - Building_SaaS_on_AWS.pptx
 
Sanket_Nasre_Simplify Modernization.pdf
Sanket_Nasre_Simplify Modernization.pdfSanket_Nasre_Simplify Modernization.pdf
Sanket_Nasre_Simplify Modernization.pdf
 
Ross Stuart_Using ML to Solve Lifes Problems.pptx
Ross Stuart_Using ML to Solve Lifes Problems.pptxRoss Stuart_Using ML to Solve Lifes Problems.pptx
Ross Stuart_Using ML to Solve Lifes Problems.pptx
 
robsable_Enhancing DevOps Practices with CloudWatch APM FINAL.pdf
robsable_Enhancing DevOps Practices with CloudWatch APM FINAL.pdfrobsable_Enhancing DevOps Practices with CloudWatch APM FINAL.pdf
robsable_Enhancing DevOps Practices with CloudWatch APM FINAL.pdf
 
Sanket_Nasre_Simplify Modernization.pdf
Sanket_Nasre_Simplify Modernization.pdfSanket_Nasre_Simplify Modernization.pdf
Sanket_Nasre_Simplify Modernization.pdf
 
Mohamed Wali_AWS Security Reference Architecture.pptx
Mohamed Wali_AWS Security Reference Architecture.pptxMohamed Wali_AWS Security Reference Architecture.pptx
Mohamed Wali_AWS Security Reference Architecture.pptx
 
Nick-Walter-HOB_Migrating_Dinosaurs.pptx
Nick-Walter-HOB_Migrating_Dinosaurs.pptxNick-Walter-HOB_Migrating_Dinosaurs.pptx
Nick-Walter-HOB_Migrating_Dinosaurs.pptx
 
Pat_Davies_AWSCostOptimization_Final.pdf
Pat_Davies_AWSCostOptimization_Final.pdfPat_Davies_AWSCostOptimization_Final.pdf
Pat_Davies_AWSCostOptimization_Final.pdf
 
MARK GAMBLE_ASC For Really Remote Edge Computing - AWS Community Day Chicago ...
MARK GAMBLE_ASC For Really Remote Edge Computing - AWS Community Day Chicago ...MARK GAMBLE_ASC For Really Remote Edge Computing - AWS Community Day Chicago ...
MARK GAMBLE_ASC For Really Remote Edge Computing - AWS Community Day Chicago ...
 
MichaelSoule-UsingJupyterNotebooks.pptx
MichaelSoule-UsingJupyterNotebooks.pptxMichaelSoule-UsingJupyterNotebooks.pptx
MichaelSoule-UsingJupyterNotebooks.pptx
 
Michal Brygidyn_CloudHackingScenarios.pdf
Michal Brygidyn_CloudHackingScenarios.pdfMichal Brygidyn_CloudHackingScenarios.pdf
Michal Brygidyn_CloudHackingScenarios.pdf
 
Kamil Kolodziejski_Structura-AWS.pptx
Kamil Kolodziejski_Structura-AWS.pptxKamil Kolodziejski_Structura-AWS.pptx
Kamil Kolodziejski_Structura-AWS.pptx
 
John Merline AWS Certification FAQ.pptx
John Merline AWS Certification FAQ.pptxJohn Merline AWS Certification FAQ.pptx
John Merline AWS Certification FAQ.pptx
 
JuliaFMorgado_Breaking_bad_habits.pptx
JuliaFMorgado_Breaking_bad_habits.pptxJuliaFMorgado_Breaking_bad_habits.pptx
JuliaFMorgado_Breaking_bad_habits.pptx
 
Jason Wadsworth - Serverless SaaS.pptx
Jason Wadsworth - Serverless SaaS.pptxJason Wadsworth - Serverless SaaS.pptx
Jason Wadsworth - Serverless SaaS.pptx
 
Joel Schuweiler_AWS IAM Identity Center (Single Sign On).pptx
Joel Schuweiler_AWS IAM Identity Center (Single Sign On).pptxJoel Schuweiler_AWS IAM Identity Center (Single Sign On).pptx
Joel Schuweiler_AWS IAM Identity Center (Single Sign On).pptx
 

Recently uploaded

Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
UK Journal
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 

Recently uploaded (20)

Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
Demystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John StaveleyDemystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John Staveley
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
 

Chicago AWS Solutions Architect Mehdy Haghy recaps the new AI/ML releases and GenAI 2023 AWS reInvent GenAI AIML recap

  • 1. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Generative AI & AI/ML on AWS Re:invent re:cap 2023 Mehdy Haghy Sr. Solutions Architect mhaghy@amazon.com
  • 2. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 2 Agenda • Generative AI Stack • Amazon Q! Your Expert. • Amazon Bedrock • Amazon Code Whisperer • Amazon SageMaker • Amazon AI Services – Gen AI Integrations • Q&A
  • 3. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 3 Generative AI Stack Inferentia Trainium SageMaker Amazon Bedrock Guardrails Agents Customization Amazon Q Amazon QuickSight Amazon Connect Amazon CodeWhisperer Knowledge Bases
  • 4. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 4 Amazon Q Y O U R G E N E R A T I V E A I A S S I S T A N T D E S I G N E D F O R W O R K > Built to be secure and private > Understands your company information, code, and system > Personalizes interactions based on your role and permissions > Engages in conversations to solve problems, generate content, and take action
  • 5. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 5 Amazon Q areas of expertise Your business Amazon QuickSight Building on AWS Amazon Connect AWS Supply Chain
  • 6. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 6 Amazon Q is your business expert (Preview) B O O S T Y O U R W O R K F O R C E P R O D U C T I V I T Y W I T H G E N E R A T I V E A I 6 > Delivers quick, accurate, and relevant answers to your business questions, securely and privately and document repositories. > Provides responses with references and citations for easy fact-checking > Respects existing access control based on user permissions > Connects to over 40 popular enterprise applications and document repositories > Enables administrators to easily apply guardrails to customize and control responses Amazon Q Amazon Q Amazon Q
  • 7. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 7 Amazon Q Builder
  • 8. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 8 Amazon Q for QuickSight
  • 9. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 9 Generative AI-powered agent assist delivers suggested responses and actions A M A Z O N Q I N C O N N E C T Detected issue Generated response Generated solution Articles and documents used to generate the response & solution
  • 10. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 10 Amazon Q in AWS Supply Chain (coming soon) 10 > Using Amazon Q in AWS Supply Chain, inventory managers, supply and demand planners, and others will be able to ask and get intelligent answers about what is happening in their supply chain, why it is happening, and what actions to take. > They will also be able to explore what-if scenarios to understand the tradeoffs between different supply chain choices.
  • 11. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 11 Amazon Bedrock The easiest way to build and scale generative AI applications on LLMs and other FMs Access a range of leading FMs via a single API Privately customize FMs with your own data Enable data security and compliance Build agents that execute complex business tasks by dynamically invoking APIs Extend the power of FMs with your data using retrieval augmented generation (RAG) Get the best price performance without managing infrastructure
  • 12. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 12 New Models in Bedrock Amazon Titan Image Generator (Preview) Amazon Titan Multimodal Embeddings Amazon Titan Text Lite and Express (now GA) Stable Diffusion XL 1.0 (Now GA) Added Llama2, Chat 13B and 70B Claude 2.1 from Anthropic
  • 13. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 13 Vector Databases for Amazon Bedrock and batch inference Vector Engine for Amazon OpenSearch Redis Enterprise Cloud Pinecone Amazon Aurora MongoDB (Coming soon) (Coming soon) Amazon Bedrock now supports batch inference Vector Search for Amazon DocumentDB
  • 14. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 14 Custom Models in Amazon Bedrock (GA) 14 • Fine-tuning and continued pre-training Deliver tailored, differentiated tail user experiences with customized FMs Fine-tune Llama 2, Command, and Titan FMs for specific tasks with labeled data Use continued pre- training to adapt Titan Text FMs to your domain with unlabeled data None of your inputs to or outputs from Amazon Bedrock will be used to train the original base models Meta—Llama2 Social media Display ads Web copy GENERATED CONTENT FINE-TUNED MODEL AMAZON BEDROCK Labeled data AMAZON S3 Copy Meta—Llama2
  • 15. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 15 Evaluate, compare, and select the best FMs for your use case Evaluate, compare, and select the best FMs for your use case in Amazon Bedrock (Preview) Automatic evaluation and human evaluation using your own work team are available today in public preview in AWS Regions US East (N. Virginia) and US West (Oregon). Human evaluation using an AWS managed team is available in public preview in AWS Region US East (N. Virginia). Preview Choose automatic or human evaluation method Curated datasets or bring your own Pre-defined and custom metrics Human evaluation report Automatic evaluation report
  • 16. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 16 Guardrails for Amazon Bedrock 16 • Implement safeguards tailored to your application requirements and responsible AI policies Preview Apply guardrails consistently across FMs including fine-tuned models and agents Configure filtering of harmful content and topics to avoid based on your responsible AI policies Redact personally identifiable information (coming soon) Guardrails for Amazon Bedrock is available today in limited preview, including Amazon Titan Text, Anthropic Claude, Meta Llama 2, AI21 Jurassic, and Cohere Command. You can also use guardrails with custom models as well as Agents for Amazon Bedrock.
  • 17. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 17 Agents and Fully managed RAG experience with Knowledge Bases (GA) Agents with improved control of orchestration and visibility into reasoning • Automatic prompt creation • Retrieval augmented generation (RAG) • Orchestrate and execute multistep tasks • Trace through the Chain of Thought (CoT) reasoning • Prompt engineering Knowledge Bases now delivers fully managed RAG experience • Fully managed support for end-to-end RAG workflow • Securely connect FMs and agents to data sources • Easily retrieve relevant data and augment prompts • Provide source attribution Agents & Knowledge bases for Amazon Bedrock are available today in AWS Regions US East (N. Virginia) and US West (Oregon).
  • 18. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 18 PartyRock and Amazon Bedrock playground Announcing PartyRock, an Amazon Bedrock Playground New Discover Apps page for PartyRock, an Amazon Bedrock Playground - https://partyrock.aws/discover
  • 19. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 19 Amazon CodeWhisperer - Your AI-powered coding companion customizations and further support Support for Infrastructure as Code (IaC), code remediation and security scanning Suggestions to remediate security and code quality issues that allows development speeds up by 28% now in Visual Studio 2022 & Amazon CodeWhisperer for command line (preview)
  • 20. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 20 Amazon SageMaker • Build, train, and deploy machine learning (ML) models at scale
  • 21. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 21 Introducing Amazon SageMaker HyperPod • Develop and train FMs continuously for weeks and months • Reduce training time for large models up by 40% Self-healing clusters reduce training time by up to 20% Resilient environment SageMaker distributed training libraries improve performance by up to 20% Streamline distributed training Control over computing environment and workload scheduling Optimized resources utilization 21
  • 22. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 22 3- Amazon SageMaker Studio Run Jupyterlab 4 application within Studio VS Code Support New setup and onboarding experience Includes latest SageMaker Distribution image Bring your own EFS volume Improved JumpStart Experience
  • 23. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 23 Amazon SageMaker Inference, Training, MLOps & IDE updates •Inference Capabilities: • Reduce model deployment costs by 50% on average. • Achieve 20% lower inference latency on average. • Deploy multiple models to the same instance for better resource utilization. • Intelligent routing of inference requests based on available instances. •Smart Data Sifting for Model Training (Preview): • Automatically inspects and evaluates training data on-the-fly. • Selectively learns from the most informative data samples. • Reduces model training time and cost by up to 35%. •Improved SDK Tooling and UX for Model Deployment: • New Python SDK library simplifies packaging and deploying ML models. • Reduces deployment process from seven steps to one. • Option for local inference. • New interactive UI experiences in SageMaker Studio for quick deployment. •Large Model Inference (LMI) with TensorRT-LLM Support: • Reduces latency by 33% on average. • Improves throughput by 60% on average. • Compatible with specific models like Llama2-70B, Falcon-40B, and CodeLlama-34B. •API Support for SageMaker Notebook Jobs: • Programmatically run notebooks as jobs using SageMaker Pipelines. • Integrated with SageMaker's ML workflow orchestration service. •Low Code Data Preparation with SageMaker Data Wrangler: • Launchable from EMR Studio. • Visual interface with 300+ transformations backed by Spark. • Analyze, clean, and create ML features without changes to existing pipelines. •Geospatial Processing Jobs: • Standardized geospatial container for accessing data catalog. • Process data with open-source algorithms or pre-trained ML models. • Visualize predictions on a map and collaborate with team members. •ML Feature Pipelines with SageMaker Feature Store: • Connect to streaming data sources and data warehouses. • Author transforms with Spark Structured Streaming. • Schedule or trigger feature processing using Amazon EventBridge rules. •Instance Expansions: • Introduction of ml.p5.48xlarge instances in the US. • Regional expansion of ml.p4d, ml.trn1, and ml.g5 instances (e.g., Tokyo, Seoul, Singapore) for SageMaker Inference. •Amazon Redshift ML (Preview) - Large Language Model Support: • Leverage pretrained LLMs in SageMaker JumpStart within Redshift ML. • Perform inferences on product feedback data for tasks like summarization, entity extraction, sentiment analysis, and classification. •New Setup and Onboarding Experience: • Individual users can create a SageMaker domain with default pre-sets. • Customizable features like Code Editor access in SageMaker Studio.
  • 24. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 24 SageMaker Clarify for FM evaluations Amazon SageMaker Clarify now supports foundation model (FM) evaluations in preview You can compare, and select FMs based on metrics such as accuracy, robustness, bias, and toxicity, in minutes. This capability is available in select regions in preview: US East (N. Virginia), US East (Ohio), US West (Oregon), Asia Pacific (Tokyo), Asia Pacific (Singapore), Europe (Frankfurt), Europe (Ireland). For additional details, see our documentation and pricing page.
  • 25. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 25 SageMaker Canvas Updates - Do more with no-code (with FMs & BI) 1- Leverage FMs for business analysis at scale Foundation models now available in Amazon SageMaker Canvas 2- Natural language instructions for data preparation, content summarization and information extraction (integration with Kendra and Bedrock/Jumpstarts) 3- Advanced configurations & Model Leaderboard & Deploy ML models built in SageMaker Canvas to SageMaker real-time endpoints 4- Amazon QuickSight announces predictive analytics using Amazon SageMaker Canvas - With this new capability, you can evolve your analytics from descriptive to predictive capabilities, enabling the entire organization with a forward-looking view of the business.
  • 26. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 26 AI Services
  • 27. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 27 AWS HealthScribe 27 A purpose-built HIPAA-eligible, generative AI service empowering healthcare software vendors to build clinical applications that automatically transcribe and summarize patient-clinician conversations. Now GA
  • 28. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 28 AWS AI Service Cards Amazon Comprehend Detect PII Amazon Transcribe Toxicity Detection Amazon Rekognition Face Liveness New Service Cards TRANSPARENCY RESOURCE TO ADVANCE RESPONSIBLE AI AWS HealthScribe
  • 29. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 29 Amazon Personalize Themes for recommendations Next-Best-Action recipe ACCELERATE YOUR BUSINESS GROWTH WITH PERSONALIZED USER EXPERIENCES Recommend the right action to the right user proactively Gen AI based Content generator identifies thematic connections between recommended items OpenSearch integration
  • 30. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 30 Amazon Lex Assisted Slot Resolution with Gen AI Conversational FAQ with Gen AI (Preview) CONVERSATIONAL AI FOR VOICE AND TEXT INTERFACES Descriptive Bot Builder using Gen AI Generate utterances using Gen AI
  • 31. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 31 Amazon Transcribe Automatic Speech Recognition for 100+ languages Automatic Language Identification for Real-Time Streams Call Analytics - Generative AI Call Summaries UNLOCKING VALUE WITH SPEECH TO TEXT
  • 32. Thank you! © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Web Services, AWS, the Powered by AWS logo, and all AWS service names used in this slide deck are trademarks of Amazon.com, Inc. or its affiliates. Mehdy Haghy Sr. Solutions Architect mhaghy@amazon.com