Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Overview of AWS Building Blocks
1. https://in.linkedin.com/in/satishraghavan Page 1
AWS Building Blocks
Source: http://www.freeimages.com/photo/visual-construct-1545402
Objective of this document
1. Provide brief overview of AWS building blocks
2. Provide reference / links for deep-dive
Snapshot
2. https://in.linkedin.com/in/satishraghavan Page 2
Compute
Service One Liner Brief Overview Product page / Short Video links
Elastic
Compute Cloud
(EC2)
Virtual servers/compute capacity
in the cloud
These are servers in
Amazon's data centers
Resizable compute
capacity on demand
Scale out or scale in
instances (30+ instances)
at present
EC2 can also reside inside
VPC
https://aws.amazon.com/documentatio
n/ec2/
Video
https://www.youtube.com/watch?v=Px
7ZPLq4AOU
https://www.youtube.com/watch?v=TO
RzO9Oc9oU&index=5&list=PLFifP_HeKx
Fk8hr-LA5XDlFfHOMosQJYp
Auto scaling Scale (launch or terminate)
Amazon EC2 instances
automatically
Scale EC2 instances
automatically
Based on user-defined
policies
Scheduled health checks
https://aws.amazon.com/documentatio
n/autoscaling/
Lambda Event base compute (Runs your
back-end code on its own based
on events or triggers)
Serverless computing
programming model
Application code should
be in one of the
programming languages
currently supported
(Node.js, Java, C# and
Python) . It uses AWS
Lambda standard runtime
Code runs in response to
events (data changes in
Amazon S3 bucket, or
DynamoDB table, HTTP
requests or invoke your
code using API calls)
https://aws.amazon.com/documentatio
n/lambda/
Video
https://www.youtube.com/watch?v=Qz
ipnZzAQEk
EC2 container
services or ECS
Run and manage Docker Container Build in scheduler or use
3
rd
party scheduler
Integrates with EBS and
ELB
Get the state of your
cluster from a centralized
service
build sophisticated
application architectures
on a microservices model
Batch/ETL Jobs,
CICD(Continuous
Integration and
Continuous Deployment)
are possible use cases
https://aws.amazon.com/documentatio
n/ecs/
Docker
https://aws.amazon.com/docker/
Container
https://aws.amazon.com/containers/
EC2 container
registry or ECR
Fully managed Docker container
registry
Resource-based Docker
repositories with IAM
integration
https://aws.amazon.com/documentatio
n/ecr/
Elastic
Beanstalk
Manage the code, not the
infrastructure
Worry about application
deployment and
https://aws.amazon.com/documentatio
n/elastic-beanstalk/
3. https://in.linkedin.com/in/satishraghavan Page 3
management applications
to AWS cloud without
worrying about the
infrastructure
Upload to code and
underlying services will
take care of
infrastructure (capacity
provisioning, handles load
balancing, app health
checks)
Major language support
includes Java, .NET,
Node.js, PHP, Ruby &
Python
Elastic Load
Balancing (ELB)
Manages the distributes the
incoming traffic
Increases fault tolerance
and availability of
application
Distributes traffic across
multiple EC2 instances
Two types: Application
Load Balancers and
Classic Load Balancers
Classic Load Balancers
supports http, https, SSL,
& TCP protocols
Application Load
Balancers supports http,
https protocols
https://aws.amazon.com/documentatio
n/elastic-load-balancing/
Video
https://www.youtube.com/watch?v=oE
cEqN8PeeI
VPC (Virtual
Private Cloud)
Launch AWS resources to VPC Once VPC is created
Amazon EC2 instance
starts running that can be
accessed from the
Internet using SSH or RDP
Elastic IP address (EIP)
enables an instance in a
VPC
https://aws.amazon.com/documentatio
n/vpc/
Video
https://www.youtube.com/watch?v=jcy
Zmj6Ywh4
Lightsail Create a development
environment
Includes everything to
launch - a virtual
machine, storage, data
transfer, DNS, IPs etc.
https://aws.amazon.com/documentatio
n/lightsail/
4. https://in.linkedin.com/in/satishraghavan Page 4
Storage
Service One Liner Brief Overview Product page
S3 (Simple
Storage
Service)
Object based Storage for the
internet
Store and retrieve data
from anywhere. It’s a key
value store
Provides intuitive
interface for interaction.
Tasks can be
accomplished using
Management Console
Not a place to install OS
or database
S3 has universal
namespace which
essentials means that it
must be unique globally
99.99% availability. It also
provides S3-IA
(infrequently accessed
data but one that
requires fast retrieval ,
reduced redundancy
storage (RRS , (data that
can be regenerated and
no impact even if it gets
lost) and Glacier (covered
below)
https://aws.amazon.com/documentat
ion/s3/
Video
https://www.youtube.com/watch?v=
VC0k-noNwOU
EBS (Elastic
Block Store)
Block level storage, fast access EBS volumes attached to
EC2 are exposed as
storage volumes
EBS recommended for
quick accessibility and
long term persistence
Provides encryption and
uses AWS Key
management services
master keys
http://docs.aws.amazon.com/AWSEC
2/latest/UserGuide/AmazonEBS.html
Video
https://www.youtube.com/watch?v=
DKftR47Ljvw
EFS (Elastic file
system)
Scalable file storage for use with
EC2
Storage capacity is elastic
and scales automatically
when files are added or
removed
Supports Network File
System version 4.1
protocol
EFS file system store data
and metadata across
multiple availability zones
Can scale to petabyte
scale
https://aws.amazon.com/documentat
ion/efs/
Glacier Cold storage for data Low cost storage services
for infrequent used data
Using Management
https://aws.amazon.com/documentat
ion/glacier/
5. https://in.linkedin.com/in/satishraghavan Page 5
Console you can create or
delete vaults
Supports REST APIs
Snowball Large data transfer (> 10 TB )
bypassing internet
Uses AWS Snowball
appliances
Import and export data
between on premise data
storage and S3
No need for buying
hardware devices
https://aws.amazon.com/documentat
ion/snowball/
Storage
Gateway
Connect on-premises software
appliance with cloud-based
storage
Use the service to store
data in the AWS Cloud
Storage Gateway offered
are file-based, volume-
based and tape-based
solutions
Scalable and cost-
effective storage
https://aws.amazon.com/documentat
ion/storage-gateway/
Database
Service One Liner Brief Overview Product page
RDS
(Relational
Database
Service)
Relational database in the cloud Set up, scale and operate
a RDBMS in the cloud
Supports MySQL,
MariaDB, PostgreSQL,
Oracle, Microsoft SQL
Server, Amazon Aurora
DB engine
https://aws.amazon.com/documentat
ion/rds/
Video
https://www.youtube.com/watch?v=
Kz1zmyHw9G0
Dynamo DB It’s a NoSQL database DynamoDB automatically
spreads the data and
traffic for the table over a
sufficient number of
servers to handle the
request
NoSQL database that
provides fast and
predictable performance
https://aws.amazon.com/documentat
ion/dynamodb/
ElastiCache Distributed in-memory cache
environments
High-performance,
scalable, and cost-
effective caching solution
Automatic detection and
recovery from cache
node failures
Memcached or Redis can
use ElastiCache
https://aws.amazon.com/documentat
ion/elasticache/
Video
https://www.youtube.com/watch?v=
8eD2eNljURE
Redshift Petabyte scale data warehouse
solution
Fully managed, petabyte-
scale data warehouse
service in the cloud
Collection of computing
resources called nodes &
clusters
https://aws.amazon.com/documentat
ion/redshift/
Video
https://www.youtube.com/watch?v=
UhQjSzdlO_g
6. https://in.linkedin.com/in/satishraghavan Page 6
Network & Content Delivery
Service One Liner Brief Overview Product page
CloudFront Distribute web content fast to end
users
Distribute static and
dynamic web content
(.html, .css, .php, images,
and media files) to end
users
Delivers content through
edge locations
Low latency for contents
Increased reliability and
availability as multiple
copies of the files (also
known as objects) are
held in multiple edge
locations across the globe
https://aws.amazon.com/documentat
ion/cloudfront/
Video
https://www.youtube.com/watch?v=
dV5qOxwAJlU
Direct Connect Dedicated connection to AWS Link internal network to
AWS Direct Connect
1 gigabit to 10 gigabit
Ethernet fiber-optic cable
https://aws.amazon.com/documentat
ion/direct-connect/
Route 53 Scalable Domain Name System
(DNS)
Port 53 handles all DNS
requests
Performs domain
registration, DNS service
and health check
Translates domain names
to IP addresses
https://aws.amazon.com/documentat
ion/route53/
Video
https://www.youtube.com/watch?v=
Nm69KMWwH7s
Security, Identity & Compliance
Service One Liner description Brief Overview Product page
Identity and
Access
management
(IAM)
Manage users and user
permissions in AWS
Securely control AWS
resources
Granular permission
Identity federation
PCI DSS compliant
Eventually consistent
https://aws.amazon.com/documentat
ion/iam/
Short Video
http://docs.aws.amazon.com/IAM/lat
est/UserGuide/introduction.html#intr
o-video
Inspector Security vulnerability assessment
service
Analyze the behavior of
your AWS resources
Identify potential security
issues
Based on AWS Shared
Responsibility Model for
security
https://aws.amazon.com/documentat
ion/inspector/
Shared Responsibility Model
https://aws.amazon.com/compliance/
shared-responsibility-model/
Artifact On-demand downloads of AWS
security & compliance
documents
Documents can be used
as guidelines to evaluate
your own cloud
architecture and assess
https://aws.amazon.com/documentat
ion/artifact/
7. https://in.linkedin.com/in/satishraghavan Page 7
the effectiveness of your
company's internal
controls.
Certificate
Manager (ACM)
Provision, deploy and manage
SSL or TLS certificates
Handles the complexity of
provisioning, deploying,
and managing SSL/TLS
certificates for your AWS-
based websites and
applications
Cannot use Amazon-
issued certificates outside
of AWS
AWS Certificate Manager
(ACM) limits apply to
each AWS region and
each AWS account.
Current default limit is
100
https://aws.amazon.com/documentat
ion/acm/
Directory
Services
It’s similar to Active Directory connect AWS services to
your existing on-premises
Microsoft Active
Directory (AD Connector),
or to set up and operate a
new directory in the AWS
cloud
Three choices - Simple
AD, Microsoft AD and AD
connector
https://aws.amazon.com/documentat
ion/directory-service/
Video
https://www.youtube.com/watch?v=
XNTsmRe8k7Q
CloudHSM Cryptographic key storage Hardware security
module (HSM) is a
hardware appliance
Cryptographic key
materials are not exposed
outside the boundary of
the appliance
PCI-DSS compliant
These are provisioned
inside VPC with an IP
address
https://aws.amazon.com/documentat
ion/cloudhsm/
KMS (Key
Management
Services)
Encryption and key management Create & control the
encryption keys for data
encryption
Integrated with many
AWS services (EBS, RDS,
Elastic transcoder, S3,
Redshift etc.)
https://aws.amazon.com/documentat
ion/kms/
Shield or WAF
(Web
application
firewall)
Web application firewall service Monitors web requests
and restricts access to
web contents
Protection against DDoS
attacks
https://aws.amazon.com/documentat
ion/waf/
8. https://in.linkedin.com/in/satishraghavan Page 8
Management Tools
Service One Liner Brief Overview Product page
CloudWatch Cloud monitoring solutions Monitor AWS resources
CloudWatch sends system
events to other AWS
resources like Lambda, SNS
Collect and track metrics
Monitor CPU usage, disk
reads etc.
https://aws.amazon.com/documentat
ion/cloudwatch/
CloudFormation Create and provision AWS
infrastructure deployments
Template based AWS
resources management
CloudFormation template is
a JSON or YAML formatted
text file
https://aws.amazon.com/documentat
ion/cloudformation/
Video
https://www.youtube.com/watch?v=
b-gwhQ6GPFQ
CloudTrail History of AWS API calls and
events
Enables logging of the AWS
API activity and events
Use CloudTrail console to
manage and monitor
https://aws.amazon.com/documentat
ion/cloudtrail/
Config Configuration view of AWS
resources
View of the configuration of
AWS resources
Notification on resource
changes
https://aws.amazon.com/documentat
ion/config/
OpsWorks Configure and operate
applications in a cloud
AWS OpsWorks for Chef
Automate provides
automated workflow for
continuous deployment and
compliance checks
https://aws.amazon.com/documentat
ion/opsworks/
Video
https://www.youtube.com/watch?v=
TPc4zdFg12M
Service Catalog Enforce compliance to
standards, manage product
lifecycles
IT services for multi-tiered
application architecture
Standardizations Self-
Service discovery,
governance and compliance
requirements
https://aws.amazon.com/documentat
ion/servicecatalog/
Health AWS Health Dashboard Provides visibility into the
state of your AWS
resources, services, and
accounts
Remediation guidance for
resources
https://aws.amazon.com/documentat
ion/health/
Management
Console
Monitor & perform AWS tasks A web application for
managing AWS
http://docs.aws.amazon.com/awscon
solehelpdocs/latest/gsg/getting-
started.html
CLI (Command
Line Interface)
One tool to manage multiple
AWS services
Unified tool to interact with
AWS
Work through command
line and automate through
scripts
https://aws.amazon.com/documentat
ion/cli/
9. https://in.linkedin.com/in/satishraghavan Page 9
Developer Tools
Service One Liner Brief Overview Product page
CodeCommit Version Control Host private Git
respositories
CodeCommit to store
anything from code to
binaries
CodeCommit repositories
are encrypted at rest as
well as in transit
https://aws.amazon.com/documentat
ion/codecommit/
CodeBuild Build service in the cloud Prepackaged build
environments (Apache
Maven, Gradle etc.)
Point to your build script
preconfigured build
environments
https://aws.amazon.com/documentat
ion/codebuild/
CodeDeploy Automate the deployment Coordinates application
deployments to Amazon
EC2 instances
https://aws.amazon.com/documentat
ion/codedeploy/
Video
https://youtu.be/Wx-ain8UryM
CodePipeline Continuous delivery service CodePipeline builds,
tests, and deploys your
code every time there is a
code change, based on
the release process
models
https://aws.amazon.com/documentat
ion/codepipeline/
Video
https://youtu.be/YxcIj_SLflw
X-Ray Analyze the behavior of
distributed applications
Collects data on requests
served by the application
Find opportunities for
optimization
X-Ray SDKs provides
interceptors (code to
trace incoming HTTP
requests)
https://aws.amazon.com/documentat
ion/xray/
10. https://in.linkedin.com/in/satishraghavan Page 10
Mobile Services
Service One Liner Brief Overview Product page
Mobile Hub Create and configure powerful
mobile app backend features
Configure, build, test
mobile apps
Use Analytics, Push
notifications,
conversational Bots, user
engagements
https://aws.amazon.com/documentat
ion/mobile-hub/
API Gateway Host a RESTful API to expose AWS
services
API Gateway consists of a
set of resources and
methods.
3rd-party app developer
to call a deployed API to
access the integrated
back-end features
RESTful API to expose
AWS Lambda functions,
HTTP endpoints as well as
other services from the
AWS family including, but
not limited to, Amazon
DynamoDB, Amazon S3
Amazon Kinesis
https://aws.amazon.com/documentat
ion/apigateway/
Cognito Create unique identities for your
users and authenticate
save mobile user data,
such as app preferences
or game state, in the AWS
Cloud without writing any
back-end code or
managing any
infrastructure
https://aws.amazon.com/documentat
ion/cognito/
Device Farm app testing service Automated testing of
apps
Remote access of devices
(phones and tablets) to
test apps. No need to
own it
https://aws.amazon.com/documentat
ion/devicefarm/
Mobile Analytics Collect, visualize and understand
app usage
Mobile Analytics web
service provides a REST
APIs
AWS Mobile SDK for
integration
https://aws.amazon.com/documentat
ion/mobileanalytics/
Pinpoint User engagement targeted push
notifications for better
user engagements
https://aws.amazon.com/documentat
ion/pinpoint/
11. https://in.linkedin.com/in/satishraghavan Page 11
Messaging
Service One Liner Brief Overview Product page
SNS (Simple
Notification
Services)
Notification services Push messaging system
Sending of messages to
subscribing endpoints
Asynchronous
communication between
Publishers or Producer
Subscribers /Consumers
(for e.g., web servers,
email, Amazon SQS
queues, Lambda
functions)
consume/receive the
notification over
supported protocols
https://aws.amazon.com/documentat
ion/sns/
Video
https://youtu.be/u5j1U3qFXDY
SES (Simple
Email services)
Email platform Build a large-scale email
solution
https://aws.amazon.com/documentat
ion/ses/
SQS (Simple
Queue Service)
Message Queueing services Decouples application
and queueing services
Asynchronous pull
Maximum payload of
256kb
Message oriented APIs
https://aws.amazon.com/documentat
ion/sqs/
Video (How Dropbox Scales Massive
Workloads Using Amazon SQS)
https://www.youtube.com/watch?v=
mP46FviScYQ
Application Services
Service One Liner Brief Overview Product page
Elastic
Transcoder
Media file conversion Convert media files into
formats compatible with
mobile, tablets, browsers
https://aws.amazon.com/documentat
ion/elastic-transcoder/
SWF (Simple
Workflow )
Coordinated Work Distribution
and task management
Coordinating tasks across
distributed application
components
Involves managing
intertask dependencies,
scheduling, and
concurrency
Task oriented APIs
https://aws.amazon.com/documentat
ion/swf/
Step Function Visual workflow JSON based language
Step through functions of
the applications
https://aws.amazon.com/documentat
ion/step-functions/
12. https://in.linkedin.com/in/satishraghavan Page 12
Migration Services
Service One Liner Brief Overview Product page
Database
migration
Services
Database migration services migrate data from your
database that is on-
premises, on an Amazon
Relational Database
Service (Amazon RDS)
Migrate data between
heterogeneous or
homogenous database
engines (Oracle to
MySQL, MySQL to
Amazon Aurora etc.)
https://aws.amazon.com/documentat
ion/dms/
Server migration
Services
Migrate from on premise to
Cloud
Automates the migration
of on-premises VMware
virtual machines to the
AWS Cloud and Amazon
EC2
https://aws.amazon.com/documentat
ion/server-migration-service/
Schema
Conversion
Convert existing database
schema to another database
engine
Convert relational OLTP
schema, or data
warehouse schema
Target can be Amazon
Aurora, Microsoft SQL
Server, MySQL,
PostgreSQL, Oracle
https://aws.amazon.com/documentat
ion/SchemaConversionTool/
Import/Export Transfer data bypassing the
internet
accelerates transferring
large amounts of data
into and out of AWS using
physical storage
appliances, bypassing the
Internet
Good choice if you have
16 terabytes (TB) or less
of data to import into
Amazon Simple Storage
Service (Amazon S3),
Amazon Glacier, or
Amazon Elastic Block
Store (Amazon EBS)
https://aws.amazon.com/documentat
ion/importexport/
Application
Discovery
Application Migration Application migration
projects by automatically
identifying servers, virtual
machines (VMs),
software, and software
dependencies
https://aws.amazon.com/documentat
ion/application-discovery/
13. https://in.linkedin.com/in/satishraghavan Page 13
Artificial Intelligence
Service One Liner Brief Overview Product page
Lex Conversational chat bots to
applications
Build conversational
interface to applications
using voice and text
Lex powers Alexa
Natural language
chatbots
https://aws.amazon.com/documentat
ion/lex/
Polly Text-to-Speech Build speech enabled
applications
Converts text to speech
Support multi-language
https://aws.amazon.com/documentat
ion/polly/
Rekognition Image analysis Detect objects, faces,
scenes in an image
Facial recognition
https://aws.amazon.com/documentat
ion/rekognition/
Machine
Learning (ML)
Algorithm base smart
applications
Finding patterns. Build
visualization tools
Wizards guides you
through the process of
creating machine learning
(ML) models
Use cases- fraud
detection, demand
forecasting, targeted
marketing & click
prediction
https://aws.amazon.com/documentat
ion/machine-learning/
Internet of Things (IoT)
Service One Liner Brief Overview Product page
IoT Two way communication
between devices and AWS
bi-directional
communication between
sensors, embedded
devices and smart
appliances to AWS
https://aws.amazon.com/documentat
ion/iot/
Other AWS Services
Area Service Product page
Game
Development
Lumberyard https://aws.amazon.com/documentation/lumberyard/
GameLift https://aws.amazon.com/documentation/gamelift/
Business
Productivity
Work Docs https://aws.amazon.com/documentation/workdocs/
Work Mail https://aws.amazon.com/documentation/workmail/
Desktop and
App Streaming
WorkSpaces https://aws.amazon.com/documentation/workspaces/
WAM (WorkSpaces Application
Manager)
https://aws.amazon.com/documentation/wam/
AppStream 2.0 https://aws.amazon.com/documentation/appstream/
14. https://in.linkedin.com/in/satishraghavan Page 14
Must Watch Videos
AWS Services Overview https://www.youtube.com/watch?v=4b3PNCSENvM
AWS Solution Architect Certification
Preparation Videos
https://www.youtube.com/watch?v=CaJCmoGIW24&list=PL4ePDvEWFtXgD5QgImq1x23sImUL26OjN
Advanced Security Best Practices Masterclass https://www.youtube.com/watch?v=zU1x5SfKEzs
How to Implement the Top Ten AWS Security
Best Practices
https://www.youtube.com/watch?v=fwvdPLu5w5A
Author:
Satish Raghavan
*Source
https://aws.amazon.com/documentation
This purpose of this document is to provide a high level overview of current AWS building blocks to kick starts ones journey towards getting AWS
Solution Architect certified. Please feel free to provide your feedback if you find any misrepresentations. Author has no intention of violating any
copyright rules. Please free to use as a reference and pass it along.
Thanks a lot for reading.
Source: https://aws.amazon.com/new/reinvent/?trk=event_reInvent2016_recap_2523
YOURSELF