[Feb AWS 201] Technical Workshop

  • 584 views
Uploaded on

AWS 201 Technical Workshop given by James Saull

AWS 201 Technical Workshop given by James Saull

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
584
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
51
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. amazon web servicesAWS 201T Workshop James Saull Solution Architect jsaull@amazon.com
  • 2. Agenda• House Keeping & Setup • Lab: Scalability and• Introduction to Amazon Web Availability Services – Snapshots• Lab: Building a Web Property – Load Balancing on AWS – Auto Scaling – Storage: S3, EBS – Security – Compute • Log Processing Scenario – Content Delivery – Logging to AWS – Relational Database – Elastic Map Reduce – Dynamo DB
  • 3. Questions ?
  • 4. House Keeping• Breaks
  • 5. Setup• aws.amazon.com• Be sure you have permissions and are active• Firefox or Chrome• Windows download Putty: www.putty.org
  • 6. Watch out for unexpected CostsWhen the Technical Workshop comes to an end, to avoidunwanted costs:• Delete your S3 objects• Destroy your CloudFront distributions• Stop or Shut Down your EC2 and RDS instancesThe customer is responsible for the resources he’s using.AWS declines any responsibility if the customer forgetsto shut down resources.
  • 7. Agenda• House Keeping & Setup Availability• Introduction to Amazon Web – Snapshots Services – Load Balancing• Lab: Building a Web Property – Auto Scaling on AWS – Security – Storage: S3, EBS • Log Processing Scenario – Compute – Logging to AWS – Content Delivery – Elastic Map Reduce – Relational Database – Dynamo DB• Lab: Scalability and
  • 8. Who is Amazon.com? 8
  • 9. Consumer businessSeller business }IT Infrastructure business[ Amazon.com’s Three Businesses ] 9
  • 10. About AmazonHow did Amazon… Web Services Deep experience in building and operating global web scale ? systems …get into cloud computing?
  • 11. – Reliable, scalable, low-cost infrastructure– Every service offers APIs– You can use only the components you need– All the important “building blocks” are provided [ What is Amazon Web Services? ]
  • 12. Region GovCloud Region[ Global AWS Infrastructure ]
  • 13. Availability Zone[ Global AWS Infrastructure ]
  • 14. Seattle South Bend New York (3) London (2)Amsterdam (2) Newark Dublin Stockholm Palo Alto Tokyo (2)SanHayward Jose Paris Frankfurt (2) Ashburn (2) Madrid Milan (2) Los Angeles (2) Jacksonville Osaka Hong Kong Dallas (2) (2) St.Louis Miami Singapore (2) Sao Paulo Edge Locations Sydney [ Global AWS Infrastructure ]
  • 15. amazon web services[ The Amazon Web Services universe ]
  • 16. Command Line API Web Console SDK[ The Amazon Web Services universe ]
  • 17. Cross Service Features CloudWatchSimple Notification Service IAM Simple WorkFlow CloudFormation Management + Elastic BeanStalk Interface[ The Amazon Web Services universe ] (some services are omitted here)
  • 18. Cross Service Features Platform Building blocks EMRRedshift SQS CloudFront Management + DynamoDB Interface RDS SES CloudSearch[ The Amazon Web Services universe ] (some services are omitted here)
  • 19. Cross Service Features Platform Building blocks Infrastructure Building Management + Blocks Interface EC2 Virtual Private Cloud S3 Glacier EBS[ The Amazon Web Services universe ] (some services are omitted here)
  • 20. Elastic Beanstalk Cross Simple WorkFlow Service CloudFormation CloudWatch Features IAM Command Line Web Console Platform APIs RDS CloudSearch SDK Building SES CloudFront SQS blocks EMR DynamoDB Infrastructure Building Management + VPC EC2 Blocks EBS S3 Interface[ The Amazon Web Services universe ] (some services are omitted here)
  • 21. Agenda• House Keeping & Setup • Lab: Scalability and• Introduction to Amazon Web Availability Services – Snapshots• Lab: Building a Web Property – Load Balancing on AWS – Auto Scaling – Storage: S3, EBS – Security – Compute • Log Processing Scenario – Content Delivery – Logging to AWS – Relational Database – Elastic Map Reduce – Dynamo DB
  • 22. LabsDuring this workshop, we will build from scratch a highly available, redundant, scalable web property on AWS.
  • 23. Our Building Blocks StorageAmazon Simple Storage Service (S3) Amazon Elastic Block Storage (Amazon EBS) Amazon S3 Bucket Amazon Elastic Volume Snapshot Block Storage (EBS)
  • 24. Our Building Blocks Compute Amazon Elastic Compute Cloud (EC2)Amazon EC2 Instance Instances AMI DB on Instance with Elastic IP Instance CloudWatch
  • 25. Our Building Blocks Content Delivery & Database Amazon Cloudfront Amazon Database Services RDS/DDB Amazon Download Streaming Edge Location Amazon RDS MySQL DB Instance DynamoDBCloudfront Distribution Distribution
  • 26. console.aws.amazon.com
  • 27. Amazon S3 Simple Storage Service• Object-based storage • Scalable (no Filesystem) • Redundancy is managed• Easily store/retrieve data transparently• Durability of 99.999999999% or • File (Object): up to 5 TB each 99.99% • HTTP, HTTPS, BitTorrent• Integrated with other AWS protocols Services 28
  • 28. Amazon S3 Redundancy Your DataData is replicated Any Amazon S3 Regionmultiple times Data DataIn case of failure,data is replicated again, Datatransparently Data Data
  • 29. Amazon S3 Use cases• Backup• Archive (Glacier)• Disaster Recovery (DR)• Content Storage• Website 30
  • 30. Lab Exercise• Create an S3 Bucket• Upload and Download a File• Check out properties and permissions
  • 31. Questions ?
  • 32. Amazon CloudFront Content Delivery Network• Delivery of content • Origin server: S3, EC2, worldwide non-AWS• Static: HTTP / HTTPS • Public/Private• Streaming: RTMP Distributions• Dynamic Content 33
  • 33. Regions, Availability Zones, Edge Locations Amsterdam Seattle South Bend New York (2) London Newark Stockholm Palo Alto Dublin TokyoSan Jose Frankfurt Paris Los Angeles (2) Ashburn Jacksonville Hong Kong Dallas St.Louis Singapore Sydney Miami Sao Paulo 38 Edge Locations in total (as of Dec 2012)
  • 34. Let’s simplify a bit: we consider only a few of them StockholmSan Jose Hong Kong Sao Paulo Your web servers in Singapore
  • 35. Content Delivery Network: How it works StockholmSan Jose Hong Kong Sao Paulo Dynamic pages (PHP, Java) (from web servers) Your web servers Static content or streaming (with CloudFront) in Singapore
  • 36. Amazon CloudFront Use cases• Accelerated web content delivery• Off-load traffic from web servers• Big spikes in traffic• Event streaming• Marketing campaigns 37
  • 37. Lab Exercise• Create a New Distribution• Test the Content
  • 38. Questions ?
  • 39. Amazon EC2 Elastic Compute Cloud• Linux / Windows virtual Image machines • VPC: Virtual Private Cloud• Elastic (scale in minutes) • Security (Groups, Keys...)• Many Instance types • VM Import (micro, small, large...) • Auto Scaling, Elastic Load• AMI: Amazon Machine Balancing 40
  • 40. Availability ZoneI see 20 Availability Zones in total (as of Feb 2013)
  • 41. EC2 Use cases• Media • Internal Applications• Hosting • Gaming• High Performance • ... Everything that Computing needs computing!• Dev & Test 42
  • 42. Lab Exercise• Create a new Security Group• Launch an Amazon EC2 instance (Linux)• Log in with SSH as ec2-user@• Install a web server• Create a simple web page• Test it on a browser: it works!• Create and attach an Elastic IP• Create an AMI from an EC2 Instance
  • 43. Demo Windows• Create a new Security Group for Windows• Launch a new EC2 instance (Windows)• Log in with RDP From Windows: log in with RDP
  • 44. Questions ?
  • 45. EBS Elastic Block Storage• Block Level Storage for use • Redundant with EC2 • Persistent• Volume: 1 GB to 1 TB • Point-in-time snapshots to• Raw unformatted block Amazon S3 device • Integration with CloudWatch• Local to an Availability Zone 46
  • 46. Demo EBS• Create a new EBS volume• Attach it to an EC2 Instance• Partition it and create a File System• Mount it; create a file in it• Create an EBS Snapshot• Create a new volume from the Snapshot
  • 47. Questions ?
  • 48. RDS Relational Database Service• Relational Database “as a • Optional: automatic Standby Service” Replica• Simple to Deploy • Optional: multiple Read-Only• Managed by the AWS team copies• MySQL, SQLServer or • Easy DB Snapshots and Oracle (as of 2012) automated backup• Scalable 49
  • 49. Demo RDS• Create a DB Instance on RDS (MySQL)• Enable Multi-AZ Deployment• Enable one Read Replica• Optional: connect to the DB Instance
  • 50. Questions ?
  • 51. Agenda• House Keeping & Setup • Lab: Scalability and• Introduction to Amazon Web Availability Services – Snapshots• Lab: Building a Web Property – Load Balancing on AWS – Auto Scaling – Storage: S3, EBS – Security – Compute • Log Processing Scenario – Content Delivery – Logging to AWS – Relational Database – Elastic Map Reduce – Dynamo DB
  • 52. Let’s take our Web Application to the next level
  • 53. EBS Elastic Load Balancer• Automatically distribute incoming traffic to multiple Amazon EC2 instances (in the same Region).• Automatic Health check• IPv6 support• Can be integrated with AutoScaling
  • 54. Questions ?
  • 55. Snapshots & AMIs• Copies of EBS Volumes• Essential to Reusability• Copy between Regions• Durability in S3
  • 56. Lab Exercise• Duplicate your entire architecture by making an AMI• Increase your availability by spreading your application across availability zones• Bring up an ELB in front of your website• Optional – Create a CNAME to the ELB
  • 57. Questions ?
  • 58. CloudWatch• Integrated across AWS platform• Nothing to install• Custom Metrics• Set Alarms• View graphs and statistics• Feeds into Autoscaling
  • 59. Lab Exercise• Check out the metrics of your web application• Set up an alarm to monitor your site
  • 60. Questions ?
  • 61. AutoScaling• Auto Up and Auto Down• Runs on CloudWatch metrics• Notifications via SNS• Spot or On-demand• No additional Fees
  • 62. AutoScaling• Launch config: AMI to be used• Autoscaling group: where/how to launch• Autoscaling policy: what should AS do• Autoscaling trigger: what will activate AS
  • 63. Demo Autoscaling JMETER
  • 64. Questions ?
  • 65. Security• Security Groups• Granular tiered secure architecture• Roles for services• Best Practices - Bastions
  • 66. Before we get started on the controls…• AWS Reports, Certifications & Accreditations • SOC 1, Type 2 report • SOC 2 report • ISO 27001 • PCI DSS Level 1 service provider • FISMA Moderate • MPAA• Look at http://aws.amazon.com/security
  • 67. A ProtectedInfrastructure: IAM VPC ACL ELB AZ SG TMG RDP WSUS AV
  • 68. Lab Exercise• Secure your website by creating a DMZ between the ELB and your application• Create a bastion host
  • 69. Questions ?
  • 70. Agenda• House Keeping & Setup • Lab: Scalability and• Introduction to Amazon Web Availability Services – Snapshots• Lab: Building a Web Property – Load Balancing on AWS – Auto Scaling – Storage: S3, EBS – Security – Compute • Log Processing Scenario – Content Delivery – Logging to AWS – Relational Database – Elastic Map Reduce – Dynamo DB
  • 71. Getting your Data into S3Console Upload S3 3rd Party Commercial Applications FTP Tsunami UDPAWS Import / Export Storage GatewayFlume S3 API CloudFront Direct Connect AWS Data Pipeline
  • 72. S3 and Big Data• Why S3?• Hadoop Overview• Hadoop on the Cloud• Hadoop File System
  • 73. Questions ?
  • 74. Introducing Apache Hadoop• Apache Hadoop • Software for distributed data analysis • Map/Reduce framework • Focus on data
  • 75. • But • Complex • Hard to setup • Cap-ex intensive • Difficult to manage
  • 76. What is Amazon Elastic MapReduce (EMR)? EMR is Hadoop in the Cloud Hadoop is an open-source framework for parallel processing huge amounts of data on a cluster of machines
  • 77. How does it work? Choose: Hadoop distribution, # of nodes, types of nodes, custom configs, Hive/Pig/etc.Put the data into S3 S3 EMR Cluster 011001101 EMR Launch the cluster using the EMR console, CLI, SDK, or APIs Get the output from S3 You can also store everything in HDFS
  • 78. DynamoDB and Big Data• What is Dynamo?• Dynamo in Big Data – Volume & Velocity
  • 79. Questions ?
  • 80. Watch out for unexpected CostsWhen the Technical Workshop comes to an end, to avoidunwanted costs:• Delete your S3 objects• Destroy your CloudFront distributions• Stop or Shut Down your EC2 and RDS instancesThe customer is responsible for the resources he’s using.AWS declines any responsibility if the customer forgetsto shut down resources.
  • 81. amazon web services Thank You James Saull Solution Architectjsaull@amazon.com