AWS 101 Event - 16 July 2013
Upcoming SlideShare
Loading in...5
×
 

AWS 101 Event - 16 July 2013

on

  • 1,352 views

Developing applications on Amazon Web Services (AWS) or moving your business into the cloud is more straightforward than you think. Whether you are a developer eager to learn new skills, a solutions ...

Developing applications on Amazon Web Services (AWS) or moving your business into the cloud is more straightforward than you think. Whether you are a developer eager to learn new skills, a solutions architect who wants to solve existing technology problems, the IT professional who wants access to cost-effective, on-demand computing resources, this slides may help you.

Amazon EC2 Demo - YouTube Recording: http://www.youtube.com/watch?v=kMExnVKhmYc&feature=youtu.be

Statistics

Views

Total Views
1,352
Views on SlideShare
1,338
Embed Views
14

Actions

Likes
1
Downloads
96
Comments
0

1 Embed 14

https://twitter.com 14

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

AWS 101 Event - 16 July 2013 AWS 101 Event - 16 July 2013 Presentation Transcript

  • AWS 101 Ianni Vamvadelis Manager, Solution Architecture
  • Agenda 10:00 - 10:45 Presentation – AWS 101 Introducing the concepts behind AWS such as utility computing and elasticity. 10:45 - 11:00 Coffee Break 11:00 - 11:45 Presentation and Demonstration Live demonstration and interactive walkthrough
  • What we are going to cover Keypairs Security groups EC2 instances Metadata service Autoscaling Amazon Machine Images S3 CloudFront Elastic Load balancer RDS Feel free to follow along on your laptops
  • background
  • Consumer Business Tens of millions of active customer accounts Eight countries: US, UK, Germany, Japan, France, Canada, China, Italy Seller Business Sell on Amazon websites Use Amazon technology for your own retail website Leverage Amazon’s massive fulfillment center network IT Infrastructure Business Cloud computing infrastructure for hosting web-scale solutions Hundreds of thousands of registered customers in over 190 countries
  • Deep experience in building and operating global web scale systems About Amazon Web Services ? …get into cloud computing? How did Amazon…
  • Over 10 years in the making Enablement of sellers on Amazon Internal need for scalable deployment environment Early forays proved developers were hungry for more
  • AWS Mission Enable businesses and developers to use web services* to build scalable, sophisticated applications. *What people now call “the cloud”
  • Not excess capacity!
  • Each day AWS adds the equivalent server capacity to power Amazon when it was a global, $5B enterprise
  • Each day AWS adds the equivalent server capacity to power Amazon when it was a global, $5B enterprise $5.2B retail business 7,800 employees A whole lot of servers 2003
  • Total Number of Objects Stored in Amazon S3 Objects in S3 2.9 Billion 14 Billion 40 Billion 102 Billion 262 Billion 762 Billion 1.7 Trillion 2 Trillion Q4 2006 Q4 2007 Q4 2008 Q4 2009 Q4 2010 Q4 2011 Q4 2012 Q2 2013 Over 1.1 Million requests per second
  • EMR Jobs 0 500,000 1,000,000 1,500,000 2,000,000 2,500,000 3,000,000 3,500,000 4,000,000 3.7 M clusters launched since May 2010
  • Utility computing
  • On demand Pay as you go Uniform Available Utility computing
  • Utility computing On demand Pay as you go Uniform Available
  • Utility computing
  • Utility computing Compute Storage Security Scaling Database Networking Monitoring Messaging Workflow DNS Load Balancing BackupCDN On demand Pay as you go Uniform Available
  • On a global footprint Region US-WEST (N. California) EU-WEST (Ireland) ASIA PAC (Tokyo) ASIA PAC (Singapore) US-WEST (Oregon) SOUTH AMERICA (Sao Paulo) US-EAST (Virginia) GOV CLOUD ASIA PAC (Sydney)
  • Availability Zone On a global footprint
  • Edge Locations Dallas(2) St.Louis Miami JacksonvilleLos Angeles (2) Palo Alto Seattle Ashburn(2) Newark New York (3) Dublin London(2) Amsterdam (2) Stockholm Frankfurt(2) Paris(2) Singapore(2) Hong Kong Tokyo (2) Sao Paulo South Bend San Jose Osaka Milan Sydney On a global footprint Madrid
  • At the end of a web service ec2-run-instances ami-b232d0db --instance-count 3 --availability-zone eu-west-1a --instance-type m1.small ec2-run-instances ami-b232d0db --instance-count 5 --availability-zone eu-west-1c --instance-type m1.medium
  • ec2-run-instances ami-b232d0db --instance-count 2 --availability-zone us-east-1b --instance-type m1.xlarge ec2-run-instances ami-b232d0db --instance-count 2 --availability-zone us-east-1d --instance-type m1.xlarge At the end of a web service
  • At the end of a web service elb-create-lb myLoadBalancer as-create-auto-scaling-group MyGroup --launch-configuration MyConfig --availability-zones eu-west-1c --min-size 2 --max-size 200 ec2-authorize default -p 80
  • and a rich Management Console
  • Elasticity
  • Traditional IT capacity Elastic capacity Capacity Time Your IT needs
  • On and Off Fast Growth Variable peaks Predictable peaks Elastic capacity
  • Elastic capacity On and Off Fast Growth Predictable peaksVariable peaks WASTE CUSTOMER DISSATISFACTION
  • Elastic cloud capacity Traditional IT capacity Your IT needs Time Capacity Elastic capacity
  • Elastic capacity Fast GrowthOn and Off Predictable peaksVariable peaks
  • 503 Service Temporarily Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
  • 503 Service Temporarily Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
  • From one instance…
  • …to thousands
  • and back…
  • Customers elasticity
  • Sunday Monday Tuesday Wednesday Thursday Friday Saturday Typical weekly traffic to Amazon.com
  • November traffic to Amazon.com November
  • November traffic to Amazon.com Provisioned capacity November
  • November traffic to Amazon.com 76% 24% Provisioned capacity November
  • November 10th 2010 Turned off last physical web server of Amazon.com
  • November 10th 2010 Turned off last physical web server of Amazon.com October 31st 2011 Turned off last web servers supporting European business
  • November traffic to Amazon.com November
  • Time: +00h <10 cores Elastic Capacity
  • Time: +24h >1500 cores Elastic Capacity
  • Time: +72h <10 cores Elastic Capacity
  • Time: +120h >600 cores Elastic Capacity
  • NumberofEC2Instances 4/12/2008 4/14/2008 4/15/2008 4/16/2008 4/18/2008 4/19/2008 4/20/20084/17/20084/13/2008 40 servers to 5000 in 3 days EC2 scaled to peak of 5000 instances “Techcrunched” Launch of Facebook modification Steady state of ~40 instances
  • The Toolbox
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Reference Model security
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Global infrastructure Regions An independent collection of AWS resources in a defined geography A solid foundation for meeting location-dependent privacy and compliance requirements
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Global infrastructure Availability Zones Designed as independent failure zones Physically separated within a typical metropolitan region
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Global infrastructure Edge Locations To deliver content to end users with lower latency A global network of edge locations Supports global DNS infrastructure (Route53) and Cloud Front CDN
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Networking Direct Connect Dedicated connection to AWS VPN Connection Secure internet connection to AWS Virtual Private Cloud Private, isolated section of the AWS Cloud Route 53 Highly available and scalable Domain Name Service
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Compute Vertical Scaling From $0.02/hr Elastic Compute Cloud (EC2) Basic unit of compute capacity Range of CPU, memory & local disk options 13 Instance types available, from micro to cluster compute Feature Details Flexible Run windows or linux distributions Scalable Wide range of instance types from micro to cluster compute Machine Images Configurations can be saved as machine images (AMIs) from which new instances can be created Full control Full root or administrator rights Secure Full firewall control via Security Groups Monitoring Publishes metrics to Cloud Watch Inexpensive On-demand, Reserved and Spot instance types VM Import/Export Import and export VM images to transfer configurations in and out of EC2
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Compute Auto-scaling Automatic provisioning of compute resources based upon demand, configuration or schedule Trigger auto- scaling policy Feature Details Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs Integrated to CloudWatch Use metrics gathered by CloudWatch to drive scaling Instance types Run auto scaling for on-demand instances and spot. Compatible with VPC as-create-auto-scaling-group MyGroup --launch-configuration MyConfig --availability-zones eu-west-1a --min-size 4 --max-size 200
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Compute Elastic Load Balancing Create highly scalable applications Distribute load across EC2 instances in multiple availability zones Feature Details Auto-scaling Automatically scales to handle request volume Available Load balance across instances in multiple availability zones Health checks Automatically checks health of instances and takes them in or out of service Session stickiness Route requests to the same instance Secure sockets layer Supports SSL offload from web and application servers with flexible cipher support Monitoring Publishes metrics to Cloud Watch
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Storage S3 - Durable storage, any object 99.999999999% durability of objects Unlimited storage of objects of any type Up to 5TB size per objectFeature Details Flexible object store Buckets act like drives, folder structures within Access control Granular control over object permissions Server-side encryption 256bit AES encryption of objects Multi-part uploads Improved throughput & control Object versioning Archive old objects and version new ones Object expiry Automatically remove old objects Access logging Full audit log of bucket/object actions Web content hosting Serve content as web site with built in page handling Notifications Receive notifications on key events Import/Export Physical device import/export service
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Storage Elastic Block Store High performance block storage device 1GB to 1TB in size Mount as drives to instances Feature Details High performance file system Mount EBS as drives and format as required Flexible size Volumes from 1GB to 1TB in size Secure Private to your instances Available Replicated within an Availability Zone Backups Volumes can be snapshotted for point in time restore Monitoring Detailed metrics captured via Cloud Watch
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Database Relational Database Service Database-as-a-Service No need to install or manage database instances Scalable and fault tolerant configurations Feature Details Platform support Create MySQL, SQL Server and Oracle RDBMS Preconfigured Get started instantly with sensible default settings Automated patching Keep your database platform up to date automatically Backups Automatic backups and point in time recovery and full DB backups Backups Volumes can be snapshotted for point in time restore Failover Automated failover to slave hosts in event of a failure Replication Easily create read-replicas of your data and seamlessly replicate data across availability zones
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Database Amazon Relational Database Service (Amazon RDS) databases stores forum threads, site content, and project configuration data. High availability Multi-AZ database deployment to handle live game metadata and user-generated content. Enterprise-grade fault tolerance for protecting customer data. By managing time-consuming database administration tasks, Amazon RDS allows SEGA to focus on business critical applications.
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Database DynamoDB Provisioned throughput NoSQL database Fast, predictable performance Fully distributed, fault tolerant architecture Feature Details Provisioned throughput Dial up or down provisioned read/write capacity Predictable performance Average single digit millisecond latencies from SSD backed infrastructure Strong consistency Be sure you are reading the most up to date values Fault tolerant Data replicated across availability zones Monitoring Integrated to Cloud Watch Secure Integrates with AWS Identity and Access Management (IAM) Elastic MapReduce Integrates with Elastic MapReduce for complex analytics on large datasets
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Application Services Amazon SQS Processing task/processing trig Processing results Amazon SQS Reliable, highly scalable, queue service for storing messages as they travel between instances Feature Details Reliable Messages stored redundantly across multiple availability zones Simple Simple APIs to send and receive messages Scalable Unlimited number of messages Secure Authentication of queues to ensure controlled access
  • Task A Task B (Auto-scaling) Task C 2 3 1 Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Application Services Feature Details Process state Maintain application state across complex workflows in a reliable and available manner Tracking Tracks executions and log process for audit purposes Consistency Ensures processing tasks are executed and duplicity of events does not occur Simple Simple Decider and Task programming model for rapid integration Simple Workflow Reliably coordinate processing steps across applications Integrate AWS and non-AWS resources Manage distributed state in complex systems
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Application Services Cloud Search Elastic search engine based upon Amazon A9 search engine Fully managed service with sophisticated feature set Scales automatically Document Server Results Search Server Feature Details Auto-scaling Automatically scales based upon request volumes and data volumes High performance In memory operation means consistently low latency for search results Sophisticated features Support for faceting, stemming, synonyms, stop words and custom rank expressions Low cost Elastic service, pay for what you use
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Application Services CloudFront World-wide content distribution network Easily distribute content to end users with low latency, high data transfer speeds, and no commitments. Feature Details Fast Multiple world-wide edge locations to serve content as close to your users as possible Integrated with other services Works seamlessly with S3 and EC2 origin servers Dynamic content Supports static and dynamic content from origin servers Streaming Supports rtmp from S3 and includes support for live streaming from Adobe FMS and Microsoft Media Server London Paris NY Served from S3 /images/* 3 Served from EC2 *.php 2 Single CNAME www.mysite.com 1
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Deployment & Admin Elastic Beanstalk One-click deployment from Eclipse, Visual Studio and Git Rapid deployment of applications All AWS resources automatically created Feature Details Platform support Containers for Java, .net and PHP Resource creation Creates load balancer, instances, autoscaling and monitoring automatically Monitoring & Logs Integrated with Cloud Watch and consolidates server logs Versioning Manage versions of applications and easily rollback deployments Notifications Receive alerts on key events Full resource access Access all underlying AWS resources as necessary
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Deployment & Admin OpsWorks DevOps focused managed application stacks Underlying Chef recipes allow for complete customisation Feature Details Platform support Chef recipes allows for community expansion for platform components such as Solr, NgniX etc Resource creation Customizable deployments, rollback, partial deployments, patch management, automatic instance scaling, and auto healing Layered Manage logical application layers and combine into stacks.
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Deployment & Admin Cloud Formation Automate creation of ‘stacks’ in a repeatable way Scripting framework for AWS resource creation Feature Details Platform support Support for AWS resources from EC2 to IAM Resource creation Creates AWS resources behind the scenes and reports on progress Declarative Specify stacks in JSON format and source control your environments Customizable Drive stack creation with paramaters
  • Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Deployment & Admin Identity & Access Management Granular control of user rights with AWS Automated granting of EC2 service rights Software Developer Kits Comprehensive support of programming models for using AWS services
  • + others Simple Email Service Simple Notification Service ElastiCache Elastic MapReduce CloudWatch …
  • Introducing Amazon Redshift Data Warehousing the AWS Way Easily and rapidly analyze petabytes of data 1/10 the cost of traditional data warehouses Automated deployment & administration Compatible with popular BI tools
  • Orchestration service for data-driven workflows Create automated and scheduled data flows Pre-integrated with AWS data sources Easily connect with 3rd party & on-premises sources AWSData Pipeline Your Analytics
  • Amazon S3 Input Bucket Amazon S3 Input Bucket Amazon S3 Output Bucket Amazon S3 Output Bucket Amazon S3 Output Bucket Transcoding Pipeline 1 Transcoding Pipeline 2 Transcoding Pipeline 4 Kindle Fire HD720p Custom    Inside a Transcoding Pipeline Transcoding Job n Transcoding Job 2 Transcoding Job 1 Amazon Elastic Transcoder
  • Security & Compliance
  • Foundation Services Compute Storage Database Networking AWS Global Infrastructure Regions Availability Zones Edge Locations Amazon Shared responsibility
  • Foundation Services Compute Storage Database Networking AWS Global Infrastructure Regions Availability Zones Edge Locations Client-side Data Encryption & Data Integrity Authentication Server-side Encryption (File System and/or Data) Network Traffic Protection (Encryption/Integrity/Identity) Platform, Applications, Identity & Access Management Operating System, Network & Firewall Configuration Customer Data Amazon Shared responsibility You
  • Certifications SOC 1 Type 2 (formerly SAS-70) ISO 27001 PCI DSS for EC2, S3, EBS, VPC, RDS, ELB, IAM FISMA Moderate Compliant Controls HIPAA & ITAR Compliant Architecture Physical Security Datacenters in nondescript facilities Physical access strictly controlled Must pass two-factor authentication at least twice for floor access Physical access logged and audited HW, SW, Network Systematic change management Phased updates deployment Safe storage decommission Automated monitoring and self-audit Advanced network protection Security standards http://aws.amazon.com/security
  • So what are we going to do today?
  • Availability Zone Region Instance
  • Availability Zone Region Instance S3
  • Availability Zone Region Instance S3 Cloud Front
  • Availability Zone Region Instance S3 Cloud Front RDS
  • Availability Zone Region Instance Instance Elastic Load Balancer Cloud Front S3 RDS
  • Availability Zone Region Instance Instance Elastic Load Balancer Cloud Front S3 Auto scaling Group RDS
  • Enjoy some coffee / tea, come back at 11:00, and we’ll build it
  • Ask questions (it will fill time when we wait for things to launch) There will be a recording of this demo on YouTube, so don’t worry if you miss anything
  • Availability Zone Region Instance Instance Elastic Load Balancer Cloud Front S3 Auto scaling Group RDS
  • Availability Zone Region Instance Instance Elastic Load Balancer Cloud Front S3 Auto scaling Group RDS
  • Availability Zone Region Instance Instance Elastic Load Balancer Cloud Front S3 Auto scaling Group RDS
  • Availability Zone Region Instance Instance Elastic Load Balancer Cloud Front S3 Auto scaling Group RDS
  • EC2 launch an instance
  • bootstrapping passing data to an instance
  • Instance request User data
  • Instance request User data Meta-data service
  • Instance request User data Instance Meta-data service
  • Script executed on launch: <script> ipconfig /all > c:ipconfig.txt netstat > c:netstat.txt </script>
  • Script executed on launch: #!/bin/sh yum -y install httpd php php-mysql chkconfig httpd on /etc/init.d/httpd start
  • key pairs secure access
  • Public Key Inserted by Amazon into each EC2 instance that you launch Private Key Downloaded and stored by you EC2 Instance Comms secured with private key
  • security groups instance firewalling
  • Security Group instance Port 80 (HTTP) Port 22 (SSH) Name Description Protocol Port range IP Address, range, or another security group
  • index.php Reads instance meta-data
  • Some php code that gets the data <?php // get the instance id $url = "http://169.254.169.254/latest/meta-data/instance-id"; $instance_id = file_get_contents($url); // get the AZ where the instance is running $url = "http://169.254.169.254/latest/meta-data/placement/availability-zone"; $zone = file_get_contents($url); // get the security group it is in $url = "http://169.254.169.254/latest/meta-data/security-groups"; $group = file_get_contents($url); // get the public DNS name $url = "http://169.254.169.254/latest/meta-data/public-hostname"; $hostname = file_get_contents($url); ?>
  • And diplays it Instance ID: <?php echo $instance_id; ?> Availability Zone: <?php echo $zone; ?> Security Group: <?php echo $group; ?>
  • ELB elastic load balancer
  • Availability Zone Availability Zone Region Availability Zone Instance Instance Instance Instance Instance Instance Elastic Load Balancer
  • Availability Zone Region Instance Instance Elastic Load Balancer Cloud Front S3 Auto scaling Group RDS
  • auto-scaling elastic server pool
  • Availability Zone Region Instance Instance Elastic Load Balancer Cloud Front S3 Auto scaling Group RDS
  • Describes what Auto Scaling will create when adding Instances AMI Instance Type Security Group Instance Key Pair Only one active launch configuration at a time Auto Scaling will terminate instances with old launch configuration first rolling update Auto Scaling managed grouping of EC2 instances Automatic health check to maintain pool size Automatically scale the number of instances by policy – Min, Max, Desired Automatic Integration with ELB Automatic distribution & balancing across AZs Parameters for performing an Auto Scaling action Scale Up/Down and by how much ChangeInCapacity (+/- #) ExactCapacity (#) ChangeInPercent (+/- %) Cool Down (seconds) Policy can be triggered by CloudWatch events Launch Configuration Auto-Scaling Group Auto-Scaling Policy
  • as-create-launch-config --image-id ami-50348c51 --instance-type t1.micro --key apac-tok --group 'Web Servers’ --launch-config 101-launch-config Create a launch configuration:
  • as-create-auto-scaling-group 101-as-group --availability-zones ap-northeast-1a --launch-configuration 101-launch-config --load-balancers myELB --max-size 5 --min-size 1 Create an auto-scaling group:
  • as-put-scaling-policy 101ScaleUpPolicy --auto-scaling-group 101-as-group --adjustment=1 --type ChangeInCapacity --cooldown 300 Create an auto-scaling policy (scale up):
  • as-put-scaling-policy 101ScaleDownPolicy --auto-scaling-group 101-as-group "--adjustment=-1" --type ChangeInCapacity --cooldown 300 Create an auto-scaling policy (scale down):
  • Availability Zone Region Instance Instance Elastic Load Balancer Cloud Front S3 Auto scaling Group RDS
  • To do this you will need: An AWS Account Auto-scaling tools installed Find out more: aws.amazon.com
  • aws.amazon.com