Your SlideShare is downloading. ×
0
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
5 things you don't know about Amazon Web Services
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

5 things you don't know about Amazon Web Services

12,023

Published on

Amazon Web Services has some not-well-known features that make it even more compelling and powerful. Watch this presentation to find out.

Amazon Web Services has some not-well-known features that make it even more compelling and powerful. Watch this presentation to find out.

Published in: Technology
2 Comments
27 Likes
Statistics
Notes
No Downloads
Views
Total Views
12,023
On Slideshare
0
From Embeds
0
Number of Embeds
21
Actions
Shares
0
Downloads
148
Comments
2
Likes
27
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Simone Brunozzi Sr. Technology Evangelist, AWS @simon things you don’t know about Amazon Web Services5 1
  • 2. 01 AWS CLI Delete a bucket and all its content with just one command 2
  • 3. 3
  • 4. aws s3 rb s3://bucket-name --force 4
  • 5. 5
  • 6. 6
  • 7. 02 AWS CLI Recursive copy of a directory and its subfolders to Amazon S3 (and vice versa) 7
  • 8. 8
  • 9. aws s3 cp MyFolder s3://bucket-name -- recursive [--region us-west-2] 9
  • 10. 03 Autoscaling Hmm... 10
  • 11. Autoscaling Amazon S3 Amazon DynamoDB Amazon ELB (Elastic Load Balancing) Amazon Glacier Amazon EB (Elastic Beanstalk) Etc. 11
  • 12. 03 EC2 Autoscaling Automatically scale out EC2 within ELB (old version) 12
  • 13. Get the AS Command Line Tools: aws.amazon.com/developertools/2535 1) Launch config:AMI to be used 2) Autoscaling group: where/how to launch 3) Autoscaling policy: what should AS do? 4) Autoscaling trigger: what will activate AS 13 AutoScaling (AS): Four main components 13
  • 14. 14 AutoScaling tools 14
  • 15. 15 CloudWatch API Tools 15
  • 16. FILE: ec2.bat @echo off set EC2_HOME=c:ec2 set PATH=%PATH%;%EC2_HOME%bin set JAVA_HOME=C:Program FilesJavajre6 "%JAVA_HOME%binjava" -version set JAVA_HOME=C:Program Files (x86)Javajre6 (Windows 7) set EC2_PRIVATE_KEY=c:ec2PrivateKey.pem set EC2_CERT=c:ec2509Certificate.pem set AWS_AUTO_SCALING_HOME=c:ec2AutoScaling set AWS_CLOUDWATCH_HOME=c:ec2CloudWatch set PATH=%PATH%;%AWS_AUTO_SCALING_HOME%bin;%AWS_CLOUDWATCH_HOME%bin 16 Windows setup 16
  • 17. FILE: .bash_profile export EC2_HOME=~/ec2 export PATH=$PATH:$EC2_HOME/bin export EC2_PRIVATE_KEY=pk-[removed].pem export EC2_CERT=cert-[removed].pem export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home/ export AWS_AUTO_SCALING_HOME=~/ec2/AutoScaling export AWS_CLOUDWATCH_HOME=~/ec2/CloudWatch export PATH=$PATH:$AWS_AUTO_SCALING_HOME/bin:$AWS_CLOUDWATCH_HOME/bin 17 Linux setup 17
  • 18. % as-version Amazon AutoScaling CLI version 1.0.61.3 (API 2011-01-01) 18 Test if it works 18
  • 19. STEP 1: Create Launch Config as-create-launch-config LC1 --image-id ami-31814f58 --instance-type m1.small [--region us-east-1] --group SG STEP 2:Autoscaling with ELB as-create-auto-scaling-group ASG1 --launch-configuration LC1 --availability- zones us-east-1b us-east-1c --min-size 2 --max-size 20 --load-balancers aws201 Check if group is created, any running EC2 as-describe-auto-scaling-groups --headers as-describe-auto-scaling-instances --headers 19 AutoScaling + ELB (1, 2) 19
  • 20. Scale in EC2 instances to ZERO, if needed as-update-auto-scaling-group ASG1 --min-size 0 --max-size 0 20 AutoScaling + ELB “Emergency break” 20
  • 21. STEP 3: Create Scaling Out Policy as-put-scaling-policy SOut1 --auto-scaling-group ASG1 --adjustment=1 --type ChangeInCapacity --cooldown 120 arn:aws:autoscaling:us- east-1:132891460296:scalingPolicy:a4a80ac4-3e3e-46a7-9b4a-72d893803618:autoSca lingGroupName/ASG1:policyName/SP1 21 AutoScaling + ELB (3) 21
  • 22. STEP 4: Create Metric Out Alarm mon-put-metric-alarm MOutA1 --comparison-operator GreaterThanThreshold -- evaluation-periods 1 --metric-name CPUUtilization --namespace "AWS/EC2" -- period 60 --statistic Average --threshold 70 --alarm-actions arn:aws:autoscaling:us- east-1:132891460296:scalingPolicy:a4a80ac4-3e3e-46a7-9b4a-72d893803618:autoSca lingGroupName/ASG1:policyName/SOut1 --dimensions "AutoScalingGroupName=ASG1" 22 AutoScaling + ELB (4) 22
  • 23. STEP 5: Create Scaling In Policy as-put-scaling-policy SInP1 --auto-scaling-group ASG1 --adjustment=-1 --type ChangeInCapacity --cooldown 120 arn:aws:autoscaling:us-east-1:132891460296:scalingPolicy:5ddb0046-a571-417f- b025-4268baf9d860:autoScalingGroupName/ASG1:policyName/SInP1 If you are using Windows, wrap the --adjustment parameter in quotation marks: "--adjustment=-1". 23 AutoScaling + ELB (5) 23
  • 24. STEP 6: Create Metric In Alarm mon-put-metric-alarm MInA1 --comparison-operator LessThanThreshold -- evaluation-periods 1 --metric-name CPUUtilization --namespace "AWS/EC2" -- period 60 --statistic Average --threshold 30 --alarm-actions arn:aws:autoscaling:us-east-1:132891460296:scalingPolicy:5ddb0046-a571-417f- b025-4268baf9d860:autoScalingGroupName/ASG1:policyName/SInP1 --dimensions "AutoScalingGroupName=ASG1" 24 AutoScaling + ELB (6) 24
  • 25. #!/bin/bash aws_instance=$(wget -q -O- http://169.254.169.254/latest/meta-data/instance- id) aws_region=$(wget -q -O- http://169.254.169.254/latest/meta-data/hostname) aws_region=${aws_region#*.} aws_region=${aws_region%%.*} aws_zone=`ec2-describe-instances $aws_instance --region $aws_region` aws_zone=`expr match "$aws_zone" ".*($aws_region[a-z])"` print "Instance: $aws_instance<br>Region: $aws_region<br>Availability Zone: $aws_zone<br><br>END" > index.html 25 Script you can use within each EC2 instance, at boot 25
  • 26. 03 EC2 Autoscaling Automatically scale out EC2 within ELB (NEW version) 26
  • 27. 27
  • 28. 04 EC2 metadata How to get details related to the EC2 machine that I’m running (PHP) 28
  • 29. 29
  • 30. 30
  • 31. 05 VPC + Cloudformation Using CF to create a 3- tier architecture within VPC. (first steps) 31
  • 32. { "AWSTemplateFormatVersion" : "2010-09-09", ! "Description": "Creates a Multi-AZ VPC with subnets for a three tier architecture", ! "Parameters" : { ! ! "VpcCidr" : { ! ! ! "Description" : "CIDR for the VPC", ! ! ! "Type" : "String", ! ! ! "MinLength": "9", ! ! ! "MaxLength": "18", ! ! ! "Default" : "10.0.0.0/16", ! ! ! "AllowedPattern" : "(d{1,3}).(d{1,3}).(d{1,3}).(d{1,3})/( d{1,2})", ! ! ! "ConstraintDescription" : "must be a valid CIDR range of the form x.x.x.x/ x." ! ! } }, ! "Resources" : { "VPC" : { "Type" : "AWS::EC2::VPC", "Properties" : { "CidrBlock" : {"Ref" : "VpcCidr"}, "Tags" : [ {"Key" : "Application", "Value" : {"Ref" : "AWS::StackId"} } ] } } ! } } 32
  • 33. { "AWSTemplateFormatVersion" : "2010-09-09", ! "Description": "Creates a Multi-AZ VPC with subnets for a three tier architecture", ! "Parameters" : { ! ! "VpcCidr" : { ! ! ! "Description" : "CIDR for the VPC", ! ! ! "Type" : "String", ! ! ! "MinLength": "9", ! ! ! "MaxLength": "18", ! ! ! "Default" : "10.0.0.0/16", ! ! ! "AllowedPattern" : "(d{1,3}).(d{1,3}).(d{1,3}).(d{1,3})/( d{1,2})", ! ! ! "ConstraintDescription" : "must be a valid CIDR range of the form x.x.x.x/ x." ! ! } }, ! "Resources" : { "VPC" : { "Type" : "AWS::EC2::VPC", "Properties" : { "CidrBlock" : {"Ref" : "VpcCidr"}, "Tags" : [ {"Key" : "Application", "Value" : {"Ref" : "AWS::StackId"} } ] } } ! } } Do you hate me? Too small? 33
  • 34. { "AWSTemplateFormatVersion" : "2010-09-09", ! "Description": "Creates a Multi-AZ VPC with subnets for a three tier architecture", ! "Parameters" : { ! ! "VpcCidr" : { ! ! ! "Description" : "CIDR for the VPC", ! ! ! "Type" : "String", ! ! ! "MinLength": "9", ! ! ! "MaxLength": "18", ! ! ! "Default" : "10.0.0.0/16", ! ! ! "AllowedPattern" : "(d{1,3}).(d{1,3}).(d{1,3}).(d{1,3})/( d{1,2})", ! ! ! "ConstraintDescription" : "must be a valid CIDR range of the form x.x.x.x/ x." ! ! } }, ! "Resources" : { "VPC" : { "Type" : "AWS::EC2::VPC", "Properties" : { "CidrBlock" : {"Ref" : "VpcCidr"}, "Tags" : [ {"Key" : "Application", "Value" : {"Ref" : "AWS::StackId"} } ] } } ! } } 34
  • 35. { "AWSTemplateFormatVersion" : "2010-09-09", ! "Description": "Creates a Multi-AZ VPC with subnets for a three tier architecture", ! "Parameters" : { ! ! "VpcCidr" : { ! ! ! "Description" : "CIDR for the VPC", ! ! ! "Type" : "String", ! ! ! "MinLength": "9", ! ! ! "MaxLength": "18", ! ! ! "Default" : "10.0.0.0/16", ! ! ! "AllowedPattern" : "(d{1,3}).(d{1,3}).(d{1,3}).(d{1,3})/( d{1,2})", ! ! ! "ConstraintDescription" : "must be a valid CIDR range of the form x.x.x.x/ x." ! ! } }, ! "Resources" : { "VPC" : { "Type" : "AWS::EC2::VPC", "Properties" : { "CidrBlock" : {"Ref" : "VpcCidr"}, "Tags" : [ {"Key" : "Application", "Value" : {"Ref" : "AWS::StackId"} } ] } } ! } } 35
  • 36. cfn-create-stack CustomerLittleNetwork --tag "Key=env; Value=temp" --template-file stack_cfn_learn_001_vpc_step1.template cfn-describe-stacks cfn-describe-stacks CustomerLittleNetwork cfn-list-stack-resources CustomerLittleNetwork 36
  • 37. 37
  • 38. 38
  • 39. cfn-update-stack CustomerLittleNetwork --template-file stack_cfn_learn_001_vpc_step2.template cfn-describe-stacks CustomerLittleNetwork cfn-list-stack-resources CustomerLittleNetwork 39
  • 40. { "AWSTemplateFormatVersion" : "2010-09-09", ! "Description": "Creates a Multi-AZ VPC with subnets for a three tier architecture", ! "Parameters" : { ! ! "VpcCidr" : { 40
  • 41. "Parameters" : { ! ! "VpcCidr" : { ! ! ! "Description" : "CIDR for the VPC", ! ! ! "Type" : "String", ! ! ! "MinLength": "9", ! ! ! "MaxLength": "18", ! ! ! "Default" : "10.0.0.0/16", ! ! ! "AllowedPattern" : "(d{1,3}).( d{1,3}).(d{1,3}).(d{1,3})/(d{1,2})", ! ! ! "ConstraintDescription" : "must be a valid CIDR range of the form x.x.x.x/x." ! ! }, 41
  • 42. ! ! "PublicSubnetCidrA" : { ! ! ! "Description" : "Subnet for public resources in the first AZ", ! ! ! "Type" : "String", ! ! ! "MinLength": "9", ! ! ! "MaxLength": "18", ! ! ! "Default" : "10.0.1.0/24", ! ! ! "AllowedPattern" : "(d{1,3}).( d{1,3}).(d{1,3}).(d{1,3})/(d{1,2})", ! ! ! "ConstraintDescription" : "must be a valid CIDR range of the form x.x.x.x/x." ! ! }, 42
  • 43. 43
  • 44. 06 CloudFormer CloudFormation templates from an existing deployment 44
  • 45. 45
  • 46. 07 Custom errors Enabling custom error pages on CloudFront 46
  • 47. 47
  • 48. 48
  • 49. 49
  • 50. 08 ZFS on Amazon EC2 How to install and run ZFS on Amazon EC2 (yes, you can!) Thanks to: Constantin Gonzalez Dougal Ballantyne (demo coming soon) 50
  • 51. What is ZFS? ZFS is a file system + logical volume manager. Features: - protection against data corruption - support for high storage capacities - integration of filesystem and volume management - snapshots and copy-on-write clones - continuous integrity checking and automatic repair - RAID-Z - native NFSv4 ACLs. OpenZFS is an Open Source implementation of it. 51
  • 52. How can I use it? Launch an EC2 instance with an AMI which supports ZFS. 1) Linux 2) OmniOS (native ZFS support based on a fork of the original OpenSolaris project that open-sourced ZFS) 3) FreeBSD (native ZFS support based on a port of the OpenSolaris ZFS code) 52
  • 53. 1) Linux Linux AMI: https://aws.amazon.com/amazon-linux-ami/ http://zfsonlinux.org/ 53
  • 54. 2) OmniOS 54
  • 55. 3) FreeBSD 55
  • 56. And now what? It works the same as with any other hardware platform: 1) Attach a few EBS volumes to your EC2 instance; 2) use them as you would use regular disks with ZFS An example follows. 56
  • 57. $ sudo modprobe zfs [attach 3 EBS volumes to the instance] $ sudo zpool create test raidz sdf sdg sdh $ sudo zpool status pool: test state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM test ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 sdf ONLINE 0 0 0 sdg ONLINE 0 0 0 sdh ONLINE 0 0 0 errors: No known data errors 57
  • 58. $ mount /dev/xvda1 on / type ext4 (rw,noatime) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) test on /test type zfs (rw,xattr) $ df -h Filesystem Size Used Avail Use% Mounted on /dev/xvda1 7.9G 1.6G 6.3G 20% / tmpfs 15G 0 15G 0% /dev/shm test 196G 0 196G 0% /test 58
  • 59. What can I do with ZFS? Many things! Here’s one example: Use an EBS Provisioned IOPS volumes as ZFS cache and log (ZIL) devices to accelerate your zpools. This works much like you would use SSDs in the real world to accelerate ZFS reads and writes. For instance, you could provision an EBS volume with 1000 IOPS and use it as a ZIL device for your zpool in order to accelerate database writes. 59
  • 60. Learn more http://pthree.org/2012/04/17/install-zfs-on-debian- gnulinux/ 60
  • 61. 09 Extra stuff Friends, nice people, cool things, etc. 61
  • 62. 62
  • 63. 63
  • 64. 64
  • 65. 65
  • 66. 66
  • 67. Thank you! 67
  • 68. Simone Brunozzi Sr. Technology Evangelist, AWS @simon things you don’t know about Amazon Web Services5 68

×