Uploaded on

AWS Summit 2014 Perth - Breakout 3 …

AWS Summit 2014 Perth - Breakout 3

Technical deep dive in to 10 AWS Cloud best practices with in-depth look at the tips and tricks of architecting on the AWS platform.

Presenter: Dean Samuels, Solutions Architect, Amazon Web Services

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
395
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Black Belt Tips on AWS Dean Samuels, Solutions Architect, Amazon Web Services © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • 2. AWS Rapid Pace of Innovation! +48! E!lastic Load! Balancing! Auto Scaling! Amazon VPC! Amazon RDS! 2009! +61! Amazon SNS! !AWS Identity ! & Access ! ! Management! Amazon Route 53! 2010! +82! Amazon SES! !AWS Elastic ! Beanstalk! !AWS ! CloudFormation! !Amazon ! ElastiCache! !AWS Direct ! Connect! GovCloud! 2011! +280! !Amazon Elastic! Transcoder! AWS OpsWorks! !Amazon ! CloudHSM! !Amazon ! AppStream! !Amazon ! CloudTrail! !Amazon ! WorkSpaces! Amazon Kinesis! 2013! +159! AWS S!torage! Gateway! !Amazon ! Dynamo DB! !Amazon ! CloudSearch! Amazon SWF! Amazon Glacier! Amazon Redshift! AWS Data ! ! Pipeline! 2012! Since inception AWS has:! ! • Released 927 new services and features ! • Introduced over 35 major new services! • Announced 45 price reductions! ! ! +24! Amazon EBS! Amazon! ! CloudFront! 2008! +270! Amazon Cognito! !Amazon Mobile! Analytics! Amazon Zocalo! 2014! *as of July 31, 2014
  • 3. Ninja Tips • Compute and Networking • Storage & Content Delivery • Deployment & Management • Security • Big Data & App Services……maybe!
  • 4. • Black Belt Tip – Route53 & Elastic Load Balancing • Cross-Zone Load Balancing • Application Meet Steve Failover via DNS Challenges • Use of AWS is starting to grow • Focus on end user experience • Minimise blast radius in event of issues • Prefers compartmentalization • Hitting AWS account limits
  • 5. • Route 53 DNS Failover ELB & Route53 • Cross-Zone Load Balancing
  • 6. Meet Steve • Black Belt Tip – Route53 & Elastic Load Balancing • Cross-Zone Load Balancing • Application Failover via DNS • Ninja Tip – VPC Peering • Trust thy neighbour! – VPC peering within an account – VPC peering between accounts Challenges • Use of AWS is starting to grow • Focus on end user experience • Minimise blast radius in event of issues • Prefers compartmentalization • Hitting AWS account limits
  • 7. VPC Peering Steve’s Shared Services VPC 10.1.0.0/16 Steve’s Workspaces VPC 192.168.0.0/20 Steve’s Enterprise Apps VPC 172.16.0.0/16 Steve’s Web Apps VPC 10.11.0.0/16 Steve’s Proxy VPC 10.20.10.0/24 Internet Dean’s WAF VPC 10.100.0.0/16 George’s Test/Dev VPC 10.10.0.0/16
  • 8. • Black Belt Tip – Storage Gateway File Shares • S3 Backed NAS – Large volume file shares, no upfront cost – On-premise or in the AWS Cloud This is Gwen Challenges • Leverages multiple storage tiers on AWS • EBS for persistent block storage • S3 for backups and serving web & media • Glacier for archiving data • But storage is starting to become costly… even on AWS • Favours the pay for what you use model with S3 rather than what you provision • Requires high performance block storage
  • 9. Next Generation Storage Corporate Data center File Servers AWS Cloud Internet or WAN SSL On-Premise AWS Storage Gateway Cache & Upload Buffer Storage Direct Attached or Storage Area Network Disks iSCSI Cached-Volumes Multi-Terabyte AWS Storage Gateway Service “Block” Volumes @ S3 Prices Encrypted & Compressed Volume Snapshots “Block” Volumes @ S3 Prices EC2 File Servers iSCSI Cached-Volumes Multi-Terabyte CIFS/ NFS Clients CIFS/ NFS EC2 Clients Third-Party options too: • Riverbed SteelStore • SoftNAS • Maginatics EC2 AWS Cached Storage Gateway Cache & Upload Buffer EBS PIOPS
  • 10. • Black Belt Tip – Storage Gateway File Shares • S3 Backed NAS – Large volume file shares, no upfront cost – On-premise or in the AWS Cloud • Ninja Tip – Instance Storage • Normally ephemeral storage – Using replication = durable storage – EBS PIOPs, General Purpose SSDs and Enhanced Networking This is Gwen Challenges • Leverages multiple storage tiers on AWS • EBS for persistent block storage • S3 for backups and serving web & media • Glacier for archiving data • But storage is starting to become costly… even on AWS • Favours the pay for what you use model with S3 rather than what you provision • Requires high performance block storage
  • 11. High Speed* & High Density* Instance storage for durable data Instance Storage with sync to EBS Instance Storage to Instance Storage to EBS General Network Traffic EBS Optimized MDADM RAID 0 array *I2 and C3 Instances: - Multiple 10s & 100’s GB SSD-based instance storage - Enhanced Networking = Higher PPS and lower jitter & latency DRBD protocol A (asynchronous) Up to 50,000 IOPs = 800MBs Data Store EC2 Instance EBS PIOPS or GP2 SSD Backed MDADM RAID 0 or 1+0 array HDD or SSD (100,000s IOPS) Enhanced Networking*
  • 12. • Black Belt Tip – AWS = Programmable Resources • AWS Support is an API • Use Resource Tags for management • Centralised logging and notification Say Hi to Felix Challenges • Still very manual deployment and configuration processes of AWS resources • Lots of human interaction • Starting to get resource sprawl – harder to manage • Not everything is supported by CloudFormation
  • 13. Everything is an API • Monitoring Your Service Limits – Via Service API • aws iam get-account-summary • aws autoscaling describe-account-limits • aws ec2 describe-account-attributes • aws ses get-send-quota – Via Trusted Advisor • aws support describe-trusted-advisor-check-result --check-id eW7HH0l7J9 --language en • Accessing Support via API – Integrate with your own management/monitoring systems – Automatically log tickets via CloudFormation
  • 14. Resource Management with Tags #!/usr/bin/ruby require 'aws-sdk' AWS.regions.sort_by(&:name).each do |region| puts region.name region.ec2.instances.each do |instance| if instance.status == :stopped and instance.tags.to_h.has_key?('DevProjectA') instance.start puts "t#{instance.id} starting" end end end for region in $(aws ec2 describe-regions --query 'Regions[*].RegionName' --output text) do Ruby SDK AWS CLI echo ${region} aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId]' --filters "Name=instance-state-name,Values=running" "Name=tag-key, Values=Uptime, Name=tag-value, Values=BusinessHoursOnly" --output text --region ${region} | xargs aws ec2 stop-instances -- instance-ids --region ${region} 2> /dev/null done
  • 15. Centralised Log Collection • CloudTrail – Get log files of API calls made on your AWS account • CloudWatch Logs – Store and Monitor OS & Application Log Files with Amazon CloudWatch • Service Logs – RDS, ELB, S3, CloudFront, EMR • Detailed Billing Reports – Cost Allocation For Customer Bills All stored in S3
  • 16. • Black Belt Tip – AWS = Programmable Resources • AWS Support is an API • Use Resource Tags for management • Centralised logging and notification • Ninja Tip – CloudFormation • Taking it to the next level! – Custom Resources Say Hi to Felix Challenges • Still very manual deployment and configuration processes of AWS resources • Lots of human interaction • Starting to get resource sprawl – harder to manage • Not everything is supported by CloudFormation
  • 17. CloudFormation Custom Resources 2 3 Custom Resource Implementation Region SQS Queue AWS CloudFormation Custom Resource Topic Auto scaling Group • Add New Resources – Including AWS resources not currently supported by CFN • Interact with the CloudFormation Workflow • Inject dynamic data into a stack • Extend the capabilities of existing resources • Data management via CloudFormation • It’s really simple if you use aws-cfn-resource-bridge – Install or fork from https://github.com/aws/aws-cfn-resource-bridge Create Parameter1:Value1 Parameter2:Value2 …. Parametern:Valuen Data Export Data Import DynamoDB Datapipeline S3 1 4 5 6 Output Parameter1:Value1 Parameter2:Value2 …. Parametern:Valuen
  • 18. CloudFormation Custom Resources 1 2 Custom Resource Implementation 4 Region SQS Queue AWS CloudFormation Custom Resource Topic Auto scaling Group • Add New Resources – Including AWS resources not currently supported by CFN • Interact with the CloudFormation Workflow • Inject dynamic data into a stack • Extend the capabilities of existing resources • Data management via CloudFormation • It’s really simple if you use aws-cfn-resource-bridge – Install or fork from https://github.com/aws/aws-cfn-resource-bridge Delete Parameter1:Value1 Parameter2:Value2 …. Parametern:Valuen Data Import Data Export 3 DynamoDB Datapipeline S3 5 Output Parameter1:Value1 Parameter2:Value2 …. Parametern:Valuen 6
  • 19. What’s up Alex? • Black Belt Tip – IAM Roles with EC2 • Don’t leave home without it! Challenges • Admin users with no MFA • Users leaving credentials in software • Users not rotating their credentials • Users not using strong password policies • Finds it hard to keep track of individual IAM identifies for users
  • 20. IAM Roles for EC2 Instances AWS Cloud Amazon S3 Amazon DynamoDB Your Application AWS IAM Your Application Your Application Your Application Auto Scaling Your Application Auto Scaling Role: RW access to objects, items and instances • Eliminates use of long-term credentials • Automatic credential rotation • Less coding – AWS SDK does all the work • Easier and more Secure! Amazon EC2
  • 21. What’s up Alex? • Black Belt Tip – IAM Roles with EC2 • Don’t leave home without it! • Ninja Tip – Limit number of IAM Users • Use IAM Roles instead – Cross-Account IAM Access – Identity Federation Challenges • Admin users with no MFA • Users leaving credentials in software • Users not rotating their credentials • Users not using strong password policies • Finds it hard to keep track of individual IAM identifies for users
  • 22. Cross-account API access dsamuel@amazon.com Acct ID: 111122223333 ec2-role {""Statement":"[" ""{" """""Action":"[" """""""ec2:StartInstances"," """""""ec2:StopInstances"" """"]," """""Effect":""Allow"," """""Resource":""*"" ""}" ]"}" squigg@amazon.com Acct ID: 123456789012 Authenticate with squigg access keys Optionally also with MFA Get temporary security credentials for ec2-role Call AWS APIs using temporary security credentials of ec2-role IAM user: squigg {""Statement":"[" ""{" """"Effect":""Allow"," """"Action":""sts:AssumeRole"," """"Resource":"" ""arn:aws:iam::111122223333:role/ec2Brole"" ""}" ]"}" {""Statement":"[" ""{" """"Effect":"Allow"," """"Principal":{"AWS":"123456789012"}," """"Action":"sts:AssumeRole"" ""}" ]"}" ec2-role trusts IAM users from the AWS account squigg@amazon.com (123456789012) Permissions assigned to squigg granting him permission to assume ec2-role in dsamuel@amazon.com account Permissions assigned to ec2-role STS Amazon EC2
  • 23. How to Keep Up to Date • AWS Podcast – http://aws.amazon.com/podcasts/aws-podcast/ • Amazon Web Services Blog – http://aws.amazon.com/blogs/aws • What’s New from AWS – http://aws.amazon.com/new • Social Media – @awscloud, /amazonwebservices, /amazonwebservices • Your Friendly Solution Architect Team – Speak to the team today at the SA booth
  • 24. Expand your skills with AWS Certification Exams Validate your proven technical expertise with the AWS platform aws.amazon.com/certification On-Demand Resources Videos & Labs Get hands-on practice working with AWS technologies in a live environment aws.amazon.com/training/ self-paced-labs Instructor-Led Courses Training Classes Expand your technical expertise to design, deploy, and operate scalable, efficient applications on AWS aws.amazon.com/training
  • 25. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.