What started as an experiment to use idle capacity has grown into the largest, most comprehensive cloud offering in the world. AWS offerings range from compute, network, storage, and backup to software like DevOps and database. My presentation slides talk about different services that Amazon is providing today and how these services operate.
2. Outline of the Talk
âť–Introduction to Cloud Computing
âť–AWS Architecture And Infrastructure
âť–AWS Services
➢ Storage Services
➢ Compute Services
➢ Database Services
➢ Networking Services
➢ Security Services
âť–Our Migration Experience to AWS
2
3. What is Cloud Computing?
âť–Cloud computing is the on-demand delivery of compute power, database
storage, applications, and other IT resources through a cloud services
platform via the internet with pay-as-you-go pricing.
3
4. Benefits of Cloud Computing
âť– Save on up-front & ongoing expense
âť– Speed, fast setup and implementation.
âť– Scalability and Global reach
âť– Low cost of entry
âť– Security and Compliance
4
8. AWS Global Infrastructure
➢A Region is a physical location in the world where we have multiple
Availability Zones.
➢Availability Zones consist of one or more discrete data centers, each with
redundant power, networking and connectivity, housed in separate facilities.
➢Edge Locations: is where end users access services at AWS. They are
located in most of the major cities around the world and are specifically used
by CloudFront(CDN) to distribute content to end user to reduce latency.
➢The AWS Cloud operates 42 Availability Zones within 16 geographic
Regions around the world.
8
12. Amazon Simple Storage Service(S3)
âť–Highly scalable object storage.
âť–Files are stored as objects and organized into
high-level folders called buckets.
âť–Pay exactly what you use
âť–Highly durable
âť–Limitlessy scalable
âť–Data in transit to and from the service is encrypted
using SSL.
âť–A bucket has a flat directory structure.
âť–You can use versioning to preserve, retrieve and
restore objects.
12
13. Bucket Properties
❖Versioning – If enabled, POST/DELETE result in the creation of new versions without
destroying the old.
❖Lifecycle – Delete or archive objects in a bucket a certain time after creation or last
access or number of versions.
❖Access Policy – Control when and where objects can be accessed.
❖Access Control – Control who may access objects in this bucket.
❖Logging – Keep track of how objects are accessed.
❖Notification – Be notified when failures occur.
13
14. Amazon Elastic Block Storage
âť–Block storage-use it as you would use a hard disk.
âť–Data persists independent of instance.
âť–Ability to backup the data via snapshots to
Amazon s3.
âť–Can store upto 16TB data per storage
âť–Attach an EBS volume to any EC2 in same AZ
âť–Different types
1.Solid-State Drives(SSD)
2. Hard disk drives(HDD)
14
15. Amazon Elastic File System (EFS)
Fully managed file system for EC2 instances
Provides standard file system semantics
Sharable across thousands of instances
between regions
Elastically grows to petabyte scale
SSD -based
Highly available and durable
NFS v4 based
Throughput and IOPS scale automatically as file
system grow.
15
16. Amazon Glacier
â—Ź An extremely low cost, easy to use data archiving
â—Ź and backup storage service in the AWS Cloud.
â—Ź Could take 3-5 hour period to retrieve data
16
20. What is Amazon EC2?
Amazon Elastic Compute Cloud (Amazon EC2)
Is a web service that provides secure, resizable
compute capacity in the cloud.
It is designed to make web-scale cloud computing
easier for developers.
It enables you to launch and manage Linux/UNIX
and Windows server instances in Amazon’s
data centers.
20
21. EC2 terminology
•Instance = One running virtual machine.
•Instance Type = hardware configuration: cores, memory, disk.
•Instance Store Volume = Temporary disk associated with instance.
•Image (AMI) = Stored bits which can be turned into instances.
•Key Pair = Credentials used to access VM from command line.
21
22. EC2 Pricing Model
Free Usage Tier
On-Demand Instances: Start and stop instances whenever you like, costs are rounded
up to the nearest hour. (Worst price)
Reserved Instances: Pay up front for one/three years in advance. (Best price)
Unused instances can be sold on a secondary market.
Spot Instances: Specify the price you are willing to pay, and instances get started and
stopped without any warning as the marked changes. (Kind of like Condor!)
22
26. RDS(Relational DataBase Service)
âť–Databases supported
➢ Aurora
➢ MySQL
➢ MariaDB
➢ Oracle
➢ MS SQL SERVER
➢ POSTGRESQL
âť–Automatic software patching
âť–General purpose and provisioned IOPS SSD
âť–Automated Backups-point-in-time recovery
âť–Multi AZ deployments-Read Replicas
âť–Network isolation.
26
27. Amazon DynamoDB
âť– Amazon DynamoDB is a fast and flexible NoSQL database
service for all applications that need consistent, single-digit
millisecond latency at any scale.
âť– It is a fully managed cloud database and supports both
document and key-value store models.
âť– Its flexible data model and reliable performance make it a
great fit for mobile, web, gaming, ad tech, IoT, and many
other applications.
❖ Automatically backup your data in three AZ’s.
27
28. Amazon ElastiCache
âť–Amazon ElastiCache is a web service that
makes it easy to deploy, operate, and scale
an in-memory data store or cache in the
cloud.
âť– to retrieve information from fast, managed,
in-memory data stores, instead of relying
entirely on slower disk-based databases.
âť–Supports two common open source engines
➢ Memcached
➢ Redis
28
29. Amazon Redshift
âť–Amazon Redshift is fast, fully managed, petabyte-scale
data warehouse service making it simple & cost
efficiently analyze all your data using existing BI tools.
âť–Amazon Redshift has a massively parallel
processing(MPP)architecture, parallelizing and
distributing SQL operations.
âť–You can use AWs Redshift to store and retrieve
processed data quickly to generate custom based
Reports.
29
32. Networking & Content Delivery
Amazon VPC Amazon
CloudFront
Amazon
Route 53
AWS Direct
Connect
Elastic Load
Balancing
â—Ź Lets you set
up a private
cloud with the
AWS .
â—Ź Is CDN service
that accelerates
delivery of your
website's, APIs,
video content or
other web assets
â—Ź Domain
Name
System
(DNS) web
service
â—Ź Establish a
dedicated
network
connection from
your premises to
AWS
. Distributes
incoming
application
traffic across
multiple EC2’s.
32
33. Amazon VPC
âť– Amazon VPC lets you provision a logically isolated section of the
Amazon Web Services (AWS) cloud where you can launch AWS
resources in a virtual network that you define.
âť– You have complete control over your virtual networking
environment, including selection of your own IP address range,
creation of subnets, and configuration of route tables and network
gateways.
âť– You can use both IPv4 and IPv6 in your VPC for secure and easy
access to resources and applications.
âť– Secure, Simple , Scalable and Reliable.
33
35. Amazon CloudFront
âť–Speed up distribution of static and dynamic
continent.
âť–Distribute media files using HTTP or HTTPS
âť–Add, update or delete object s and submit data
from web forms.
âť–Use live streaming to stream an event in real time.
35
37. AWS Route 53
âť–Amazon Route 53 is a highly available and
scalable DNS web service.
âť–Use advanced routing policies along with health
checks and failovers to ensure your application is
always available to end users and provides and
best experience.
37
38. AWS Direct Connect
AWS Direct Connect makes it easy to establish a dedicated network
connection from your premises to AWS without involving Internet.
Direct Connect is also useful for customers whose corporate governance
prohibits the use Of the public Internet for transferring sensitive data
because of compliance concerns.
Can be used with all AS Services including (EC2, VPC, S3, DynamoDB)
38
42. Identity & Access Management
âť–AWS Identity and Access Management (IAM) enables
you to securely control access to AWS services and
resources for your users.
âť– Using IAM you can create and manage AWS users
and groups, and use permissions to allow and deny
their access to AWS resources.
âť–IAM is a feature of your AWS account offered at no
additional charge
âť–Main components Users, Groups, Roles and Policies.
42
43. Amazon Elastic BeanStalk
âť–Elastic Beanstalk can quickly deploy and manage web
applications in the AWS cloud.
âť–Amazon's Elastic Beanstalk provisions necessary
infrastructure resources such as load balancer, auto
scaling group, security groups, databases (optional).
âť–Provides Web servers and worker tiers and provides a
unique name for your application (eg:
yourapp.elasticbeanstalk.com)
43
45. AWS CloudWatch
âť–Amazon CloudWatch is a monitoring service for AWS cloud
resources and application you run in AWS.
âť–You can use Amazon CloudWatch to collect and track
metrics, collect and monitor log files, set alarms, and
automatically react to changes in your AWS resources.
âť–Amazon CloudWatch can monitor AWS resources such as
Amazon EC2 instances, Amazon DynamoDB tables, and
Amazon RDS DB instances, as well as custom metrics
generated by your applications and services, and any log
files your applications generate.
45
48. AWS Import,Delivery,Export Services
AWS Import/Export:
â—Ź Is a service that accelerates
transferring data into and out of
AWS using physical storage
appliances, bypassing the Internet.
AWS Snowball Edge:
â—Ź Is a 100TB data transfer
device with on-board storage
and compute capabilities.
AWS Snowmobile:
â—Ź Is an Exabyte-scale data
transfer service used to move
extremely large amounts of data
to AWS. You can transfer up to
100PB per Snowmobile,
48
49. private static final int THREADPOOLSIZE = 100;
private static final int DIVISORVALUE = 100;
for (Integer volumeid : listOfActiveVolumes) {
ExecutorService executor = Executors.newFixedThreadPool(THREADPOOLSIZE);
for (int i = 0; i < THREADPOOLSIZE; i++) {
String query = "select distinct prodid from tableProds " +
" where volumeid in ( " + volumeid + "n" +
"nn)" +
" and prodid % " + DIVISORVALUE + "=" + i + " order by prodid desc";
ArrayList<Integer> listOfProds = runQuery(query);
Runnable worker = new doCopyToAWS(listOfProds );
executor.execute(worker);
try {
executor.shutdown();
executor.awaitTermination(5, TimeUnit.SECONDS);
} catch (InterruptedException e) {
logger.error("tasks interrupted");
} finally {
if (!executor.isTerminated()) {
logger.error("cancel non-finished tasks");
}
executor.shutdownNow();
logger.info("Shutdown Finished ."); 49
Editor's Notes
These Availability Zones offer you the ability to operate production applications and databases which are more highly available, fault tolerant and scalable than would be possible from a single data center.
AWS GovCloud (2)
US West
Oregon (3),
Northern California (3)
US East
Northern Virginia (5), Ohio (3)
Canada
Central (2)
South America
SĂŁo Paulo (3)
Europe
Ireland (3), Frankfurt (2), London (2)
Asia Pacific
Singapore (2), Sydney (3), Tokyo (3), Seoul (2), Mumbai (2)
China
Beijing (2)
Region Coming Soon
New Region (coming soon)
Paris
Ningxia
-EBS can’t be accessed across regions.
Works with standard operatins system APIs
But we could set up different retrieval poly for each AWS region.
Free Tier Only, Max Retrieval Rate, or No Retrieval Limit.
To use file gateway storage, you download a virtual machine image for the file storage gateway and activate it from the AWS Management Console or the storage gateway API.
The Storage Gateway virtual appliance connects directly to your local infrastructure as a file server, as a local disk volume, or as a virtual tape library (VTL). This seamless connection makes it simple for organizations to augment existing on-premises storage investments with the high scalability, extreme durability and low cost of AWS cloud storage.
•A bucket is a container for objects and describes location, logging, accounting, and access control. A bucket can hold any number of objects, which are files of up to 5TB. A bucket has a name that must be globally unique.
•A bucket has a flat directory structure (despite the appearance given by the interactive web interface.)
1.Determines the hardware of the host computer used for your instance.
2. Each instance type offers different compute, memory, and storage capabilities and are grouped in instance families based on these capabilities.
You don’t have to worry about database maintenance or administration.
Fast response times and ability to easily scale with demand.
Application’s needing a data structure server will find the Redis engine more useful to provide
Features leaderboard, counting, session management and tracking hard to do with relational databases.
Many customers like Memcached’s simplicity.
AWS SnowMobile; 45-foot long ruggedized shipping container, pulled by a semi-trailer truck.