AWS 101 Event December 2013
Upcoming SlideShare
Loading in...5
×
 

AWS 101 Event December 2013

on

  • 1,601 views

Developing applications on Amazon Web Services (AWS) or moving your business into the cloud is more straightforward than you think. Whether you are a developer eager to learn new skills, a solutions ...

Developing applications on Amazon Web Services (AWS) or moving your business into the cloud is more straightforward than you think. Whether you are a developer eager to learn new skills, a solutions architect who wants to solve existing technology problems, the IT professional who wants access to cost-effective, on-demand computing resources, this slides may help you.

Statistics

Views

Total Views
1,601
Views on SlideShare
1,601
Embed Views
0

Actions

Likes
0
Downloads
35
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

AWS 101 Event December 2013 AWS 101 Event December 2013 Presentation Transcript

  • AWS 101 Alistair McLaurin AWS Solution Architecture
  • Agenda 10:00 - 10:45 Presentation – AWS 101 Introducing the concepts behind AWS, such as utility computing and elasticity. 10:45 - 11:00 Coffee Break 11:00 - 11:45 Presentation and Demonstration Live demonstration and interactive walkthrough
  • What we are going to cover Keypairs Amazon Machine Images Security groups S3 EC2 instances CloudFront Metadata service Elastic Load balancer Autoscaling RDS Feel free to follow along on your laptops
  • background
  • Consumer Business Tens of millions of active customer accounts 8 countries: US, UK, Germany, Japan, France, Canada, China, Italy Seller Business IT Infrastructure Business Sell on Amazon websites Cloud computing infrastructure for hosting web-scale solutions Use Amazon technology for your own retail website Leverage Amazon’s massive fulfillment center network Hundreds of thousands of registered customers in over 190 countries
  • About Amazon Web Services How did Amazon… ? Deep experience in building and operating global web scale systems …get into cloud computing?
  • AWS Mission Enable businesses and developers to use web services* to build scalable, sophisticated applications. *What people now call “the cloud”
  • Not excess capacity!
  • Each day AWS adds the equivalent server capacity to power Amazon when it was a global, $7B enterprise
  • Objects in S3 2 Trillion 1.7 Trillion Over 1.1 Million requests per second 762 Billion 262 Billion 2.9 Billion 14 Billion 40 Billion Q4 2006 Q4 2007 Q4 2008 102 Billion Q4 2009 Q4 2010 Q4 2011 Q4 2012 Total Number of Objects Stored in Amazon S3 Q2 2013
  • utility computing
  • Utility computing On demand Uniform Pay as you go Available
  • Utility computing On demand Uniform Pay as you go Available
  • Utility computing
  • Utility computing On demand Compute Security DNS Scaling CDN Backup Database Storage Load Balancing Workflow Uniform Pay as you go Monitoring Networking Messaging Available
  • On a global footprint Region US-WEST (N. California) EU-WEST (Ireland) ASIA PAC (Tokyo) GOV CLOUD US-EAST (Virginia) US-WEST (Oregon) ASIA PAC (Singapore) SOUTH AMERICA (Sao Paulo) ASIA PAC (Sydney)
  • At the end of a web service ec2-run-instances ami-b232d0db --instance-count 3 --availability-zone eu-west-1a --instance-type m1.small ec2-run-instances ami-b232d0db --instance-count 5 --availability-zone eu-west-1c --instance-type m1.medium
  • and a rich Management Console
  • elasticity
  • Elastic capacity Traditional IT capacity Capacity Time Your IT needs
  • Elastic capacity On and Off Fast Growth Variable peaks Predictable peaks
  • Elastic capacity WASTE On and Off Fast Growth Variable peaks Predictable peaks CUSTOMER DISSATISFACTION
  • Elastic capacity On and Off Fast Growth Variable peaks Predictable peaks
  • From one instance…
  • …to thousands
  • and back…
  • exploiting elasticity
  • Typical weekly traffic to Amazon.com Sunday Monday Tuesday Wednesday Thursday Friday Saturday
  • November traffic to Amazon.com November
  • November traffic to Amazon.com Provisioned capacity November
  • November traffic to Amazon.com 76% Provisioned capacity November 24%
  • November 10th 2010 Turned off last physical web server of Amazon.com
  • November 10th 2010 Turned off last physical web server of Amazon.com October 31st 2011 Turned off last web servers supporting European business
  • November traffic to Amazon.com November
  • 40 servers to 5000 in 3 days Number of EC2 Instances EC2 scaled to peak of 5000 instances “Techcrunched” Launch of Facebook modification Steady state of ~40 instances 4/12/2008 4/13/2008 4/14/2008 4/15/2008 4/16/2008 4/17/2008 4/18/2008 4/19/2008 4/20/2008
  • the toolbox
  • Reference Model security Deployment & Administration App Services Compute Storage Database Networking AWS Global Infrastructure
  • Global infrastructure Deployment & Administration App Services Compute Storage Database Regions An independent collection of AWS resources in a Networking defined geography A solid foundation for meeting location-dependent AWS Global Infrastructure privacy and compliance requirements
  • Global infrastructure Deployment & Administration App Services Compute Storage Database Availability Zones Designed as independent failure zones Networking Physically separated within a typical metropolitan region AWS Global Infrastructure
  • Global infrastructure Deployment & Administration App Services Compute Storage Database Edge Locations To deliver content to end users with lower latency Networking A global network of edge locations Supports global DNS infrastructure (Route53) and AWS Global Infrastructure Cloud Front CDN
  • Networking Direct Connect Dedicated connection to AWS VPN Connection Secure internet connection to AWS Deployment & Administration App Services Compute Storage Virtual Private Cloud Private, isolated section of the AWS Cloud Database Networking Route 53 Highly available and scalable Domain Name Service AWS Global Infrastructure
  • Compute Vertical Scaling From $0.02/hr Elastic Compute Cloud (EC2) Basic unit of compute capacity Range of CPU, memory & local disk options 13 Instance types available, from micro to cluster compute Feature Details App Services Run windows or linux distributions Scalable Deployment & Administration Flexible Wide range of instance types from micro to cluster compute Machine Images Full control Compute Storage Database Secure Configurations can be saved as machine images (AMIs) from which new instances can be created Full root or administrator rights Full firewall control via Security Groups AWS Global Infrastructure Monitoring Publishes metrics to Cloud Watch Inexpensive Networking On-demand, Reserved and Spot instance types VM Import/Export Import and export VM images to transfer configurations in and out of EC2
  • Compute Trigger autoscaling policy as-create-auto-scaling-group MyGroup --launch-configuration MyConfig --availability-zones eu-west-1a --min-size 4 --max-size 200 Deployment & Administration Auto-scaling Automatic provisioning of compute resources based App Services upon demand, configuration or schedule Compute Storage Database Feature Control Networking Details Define minimum and maximum instance pool sizes and when scaling and cool down occurs Use metrics gathered by CloudWatch to drive scaling Instance types AWS Global Infrastructure Integrated to CloudWatch Run auto scaling for on-demand instances and spot. Compatible with VPC
  • Compute Elastic Load Balancing Create highly scalable applications Distribute load across EC2 instances in multiple availability zones Deployment & Administration Feature Auto-scaling App Services Available Compute Storage Database Health checks Session stickiness Networking AWS Global Infrastructure Secure sockets layer Monitoring Details Automatically scales to handle request volume Load balance across instances in multiple availability zones Automatically checks health of instances and takes them in or out of service Route requests to the same instance Supports SSL offload from web and application servers with flexible cipher support Publishes metrics to Cloud Watch
  • Storage S3 - Durable storage, any object 99.999999999% durability of objects Unlimited storage of objects of any type Feature Details Up to 5TB size per object Flexible object store Access control Deployment & Administration Server-side encryption Multi-part uploads Buckets act like drives, folder structures within Granular control over object permissions 256bit AES encryption of objects Improved throughput & control App Services Object versioning Compute Storage Database Object expiry Access logging Networking Web content hosting Notifications AWS Global Infrastructure Import/Export Archive old objects and version new ones Automatically remove old objects Full audit log of bucket/object actions Serve content as web site with built in page handling Receive notifications on key events Physical device import/export service
  • Storage Elastic Block Store High performance block storage device 1GB to 1TB in size Mount as drives to instances Feature High performance file system Deployment & Administration Flexible size Secure App Services Available Compute Storage Database Backups Monitoring Networking AWS Global Infrastructure Details Mount EBS as drives and format as required Volumes from 1GB to 1TB in size Private to your instances Replicated within an Availability Zone Volumes can be snapshotted for point in time restore Detailed metrics captured via Cloud Watch
  • Database Relational Database Service Database-as-a-Service No need to install or manage database instances Scalable and fault tolerant configurations Deployment & Administration Feature Platform support App Services Preconfigured Details Create MySQL, SQL Server and Oracle RDBMS Get started instantly with sensible default settings Automated patching Storage Database Networking AWS Global Infrastructure Backups Automatic backups and point in time recovery and full DB backups Backups Volumes can be snapshotted for point in time restore Failover Compute Keep your database platform up to date automatically Automated failover to slave hosts in event of a failure Replication Easily create read-replicas of your data and seamlessly replicate data across availability zones
  • Database Amazon Relational Database Service (Amazon RDS) databases stores forum threads, site content, and project configuration data. High availability Multi-AZ database deployment to handle live game metadata and user-generated content. Deployment & Administration App Services Compute Storage Database Networking AWS Global Infrastructure Enterprise-grade fault tolerance for protecting customer data. By managing time-consuming database administration tasks, Amazon RDS allows SEGA to focus on business critical applications.
  • Database DynamoDB Provisioned throughput NoSQL database Fast, predictable performance Fully distributed, fault tolerant architecture Feature Deployment & Administration Provisioned throughput Details Dial up or down provisioned read/write capacity Predictable performance Strong consistency App Services Compute Storage Database Average single digit millisecond latencies from SSD backed infrastructure Be sure you are reading the most up to date values Fault tolerant Monitoring Networking Secure Elastic MapReduce AWS Global Infrastructure Data replicated across availability zones Integrated to Cloud Watch Integrates with AWS Identity and Access Management (IAM) Integrates with Elastic MapReduce for complex analytics on large datasets
  • Database RDS Dynamo DB Redshift Deployment & Administration App Services Compute Storage Database Redshift Managed Massively Parallel Petabyte Scale Data Networking AWS Global Infrastructure Warehouse Streaming Backup/Restore to S3 Extensive Security 2 TB -> 1.6 PB
  • Application Services CloudFront 3 Served from S3 /images/* World-wide content distribution network Easily distribute content to end users with low latency, high data transfer speeds, and no commitments. 2 London Served from EC2 *.php Paris Deployment & Administration 1 Single CNAME NY www.mysite.com App Services Feature Compute Storage Database Networking AWS Global Infrastructure Fast Details Multiple world-wide edge locations to serve content as close to your users as possible Integrated with other services Works seamlessly with S3 and EC2 origin servers Dynamic content Supports static and dynamic content from origin servers Streaming Supports rtmp from S3 and includes support for live streaming from Adobe FMS and Microsoft Media Server
  • Application Services Amazon SQS Processing results Reliable, highly scalable, queue service for storing messages as they Amazon SQS travel between instances Deployment & Administration Processing task/processing trig App Services Feature Compute Storage Database Reliable Simple Networking AWS Global Infrastructure Scalable Secure Details Messages stored redundantly across multiple availability zones Simple APIs to send and receive messages Unlimited number of messages Authentication of queues to ensure controlled access
  • Application Services Simple Workflow 1 Reliably coordinate processing steps across applications 2 Task A Integrate AWS and non-AWS resources Manage distributed state in complex systems 3 Task B (Auto-scaling) Deployment & Administration Task C App Services Feature Details Process state Tracking Compute Storage Maintain application state across complex workflows in a reliable and available manner Tracks executions and log process for audit purposes Database Networking Consistency Simple AWS Global Infrastructure Ensures processing tasks are executed and duplicity of events does not occur Simple Decider and Task programming model for rapid integration
  • Application Services Cloud Search Document Server Elastic search engine based upon Amazon A9 search engine Fully managed service with sophisticated feature set Search Server Scales automatically Deployment & Administration App Services Results Feature Auto-scaling Compute Storage Database High performance Networking Sophisticated features Low cost AWS Global Infrastructure Details Automatically scales based upon request volumes and data volumes In memory operation means consistently low latency for search results Support for faceting, stemming, synonyms, stop words and custom rank expressions Elastic service, pay for what you use
  • Deployment & Admin Elastic Beanstalk One-click deployment from Eclipse, Visual Studio and Git Rapid deployment of applications All AWS resources automatically created Deployment & Administration Feature Platform support App Services Details Containers for Java, .net and PHP Resource creation Compute Storage Database Creates load balancer, instances, autoscaling and monitoring automatically Monitoring & Logs Integrated with Cloud Watch and consolidates server logs Versioning Networking Notifications AWS Global Infrastructure Full resource access Manage versions of applications and easily rollback deployments Receive alerts on key events Access all underlying AWS resources as necessary
  • Deployment & Admin OpsWorks DevOps focused managed application stacks Underlying Chef recipes allow for complete customisation Feature Deployment & Administration App Services Compute Storage Platform support Resource creation Database Networking AWS Global Infrastructure Layered Details Chef recipes allows for community expansion for platform components such as Solr, NgniX etc Customizable deployments, rollback, partial deployments, patch management, automatic instance scaling, and auto healing Manage logical application layers and combine into stacks.
  • Deployment & Admin Cloud Formation Automate creation of ‘stacks’ in a repeatable way Scripting framework for AWS resource creation Deployment & Administration Feature Platform support App Services Details Support for AWS resources from EC2 to IAM Resource creation Compute Storage Database Creates AWS resources behind the scenes and reports on progress Declarative Specify stacks in JSON format and source control your environments Customizable Networking AWS Global Infrastructure Drive stack creation with paramaters
  • Deployment & Admin Identity & Access Management Granular control of user rights with AWS Automated granting of EC2 service rights Software Developer Kits Comprehensive support of programming models for Deployment & Administration App Services Compute Storage Database Networking AWS Global Infrastructure using AWS services
  • + others Simple Email Service Simple Notification Service ElastiCache (Memcache & Redis) Elastic MapReduce CloudWatch …and more to come!
  • security & compliance
  • Shared responsibility Amazon Foundation Services Compute Storage Database Networking Availability Zones AWS Global Infrastructure Edge Locations Regions
  • Shared responsibility You Customer Data Platform, Applications, Identity & Access Management Operating System, Network & Firewall Configuration Client-side Data Encryption & Data Integrity Authentication Server-side Encryption (File System and/or Data) Network Traffic Protection (Encryption/Integrity/Identity) Amazon Foundation Services Compute Storage Database Networking Availability Zones AWS Global Infrastructure Edge Locations Regions
  • Security standards Certifications Physical Security HW, SW, Network SOC 1 Type 2 (formerly SAS-70) Datacenters in nondescript facilities Systematic change management ISO 27001 Physical access strictly controlled Phased updates deployment Must pass two-factor authentication at least twice for floor access Safe storage decommission PCI DSS for EC2, S3, EBS, VPC, RDS, ELB, IAM FISMA Moderate Compliant Controls HIPAA & ITAR Compliant Architecture Physical access logged and audited Automated monitoring and self-audit Advanced network protection http://aws.amazon.com/security
  • So what are we going to build today?
  • Instance Availability Zone Region
  • Instance Availability Zone S3 Region
  • Cloud Front Instance Availability Zone S3 Region
  • Cloud Front S3 Instance RDS Availability Zone Region
  • Cloud Front Elastic Load Balancer Instance Instance S3 RDS Availability Zone Region
  • Cloud Front Elastic Load Balancer Instance Instance S3 Auto scaling Group RDS Availability Zone Region
  • Enjoy some coffee / tea, come back at 11:00, and we’ll build it
  • Ask questions (it will fill time when we wait for things to launch) There will be a recording of this demo on YouTube, so don’t worry if you miss anything
  • Instance Availability Zone Region
  • EC2 launch an instance
  • bootstrapping passing data to an instance
  • Instance request User data
  • Instance request User data Meta-data service
  • Instance request User data Meta-data service Instance
  • Script executed on launch: <script> ipconfig /all > c:ipconfig.txt netstat > c:netstat.txt </script>
  • Script executed on launch: #!/bin/sh yum -y install httpd php php-mysql chkconfig httpd on /etc/init.d/httpd start
  • security groups instance firewalling
  • Port 22 (SSH) Port 80 (HTTP) Security Group instance Name Description Protocol Port range IP Address, range, or another security group
  • key pairs secure access
  • Public Key Inserted by Amazon into each EC2 instance that you launch EC2 Instance Comms secured with private key Private Key Downloaded and stored by you
  • index.php Reads instance meta-data
  • Some php code that gets the data <?php // get the instance id $url = "http://169.254.169.254/latest/meta-data/instance-id"; $instance_id = file_get_contents($url); // get the AZ where the instance is running $url = "http://169.254.169.254/latest/meta-data/placement/availability-zone"; $zone = file_get_contents($url); // get the security group it is in $url = "http://169.254.169.254/latest/meta-data/security-groups"; $group = file_get_contents($url); // get the public DNS name $url = "http://169.254.169.254/latest/meta-data/public-hostname"; $hostname = file_get_contents($url); ?>
  • And diplays it Instance ID: <?php echo $instance_id; ?> Availability Zone: <?php echo $zone; ?> Security Group: <?php echo $group; ?>
  • Instance Availability Zone S3 Region
  • Cloud Front Instance Availability Zone S3 Region
  • CloudFront Global CDN
  • Cloud Front S3 Instance RDS Availability Zone Region
  • Amazon RDS Managed Relational DB
  • Cloud Front Elastic Load Balancer S3 Instance RDS Availability Zone Region
  • ELB elastic load balancer
  • Elastic Load Balancer Instance Instance Availability Zone Instance Instance Availability Zone Region Instance Instance Availability Zone
  • Cloud Front Elastic Load Balancer Instance Instance S3 Auto scaling Group RDS Availability Zone Region
  • auto-scaling elastic server pool
  • Launch Configuration Auto-Scaling Group Auto-Scaling Policy Describes what Auto Scaling will create when adding Instances Auto Scaling managed grouping of EC2 instances Parameters for performing an Auto Scaling action AMI Instance Type Security Group Instance Key Pair Automatic health check to maintain pool size Scale Up/Down and by how much Automatically scale the number of instances by policy – Min, Max, Desired ChangeInCapacity (+/- #) ExactCapacity (#) ChangeInPercent (+/- %) Automatic Integration with ELB Cool Down (seconds) Only one active launch configuration at a time Auto Scaling will terminate instances with old launch configuration first rolling update Automatic distribution & balancing across AZs Policy can be triggered by CloudWatch events
  • Create a launch configuration: aws autoscaling create-launchconfiguration --launch-configuration-name aws-101 -image-id ami-d79b78a0 --security-groups ssh-only --instance-type m1.small --region eu-west-1
  • Create an auto-scaling group: aws autoscaling create-auto-scaling-group -auto-scaling-group-name aws-101-sg --region eu-west-1 --launch-configuration-name aws-101 --min-size 0 --max-size 3 --desired-capacity 0 --availability-zones eu-west-1a eu-west-1b eu-west-1c --load-balancer-names aws-101
  • Create an auto-scaling policy (scale up): aws autoscaling put-scaling-policy --region eu-west-1 --auto-scaling-group-name aws-101-sg --policy-name cpu-up --scaling-adjustment 1 --adjustment-type ChangeInCapacity --cooldown 0 { "PolicyARN": "arn:aws:autoscaling:eu-west1:887210671223:scalingPolicy:47a05c37-dc0a-4366-89916272cc1816fd:autoScalingGroupName/aws-101sg:policyName/cpu-up" }
  • Create a scaling Trigger from CloudWatch aws cloudwatch put-metric-alarm --alarm-name aws-101-scale-up --metric-name CPUUtilization --namespace "AWS/EC2" --statistic Average --threshold 80 --period 60 --comparison-operator GreaterThanOrEqualToThreshold --evaluation-periods 2 --alarm-actions arn:aws:autoscaling:eu-west1:887210671223:scalingPolicy:47a05c37-dc0a4366-89916272cc1816fd:autoScalingGroupName/aws-101sg:policyName/cpu-up --region eu-west-1
  • Cloud Front Elastic Load Balancer Instance Instance S3 Auto scaling Group RDS Availability Zone Region
  • To do this you will need: An AWS Account Auto-scaling tools installed Find out more: aws.amazon.com
  • aws.amazon.com