Cloud Computing Reference Guide
•
24 likes•1,879 views
The Cloud as a Platform Keynote Presentation delivered at Cloud Connections Conference (DevConnections) April 19, 2011 by Jinesh Varia, Technology Evangelist, Amazon
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Cloud Computing Reference Guide
Similar to Cloud Computing Reference Guide (20)
More from Amazon Web Services
More from Amazon Web Services (20)
Recently uploaded
Recently uploaded (20)
Cloud Computing Reference Guide
- 2. Published by Wiley (March 29, 2011) 25 Chapters contributed by various authors around the world Reference guide for Practitioners, new researchers, developers architects, and Graduate Students (textbook) Captures the state of the art in cloud computing technologies and applications
- 5. And pace accelerates in 2010…. » Amazon RDS Read Replicas » Suse EC2 Linux » Amazon SNS Console » Amazon ELB HTTPS » AWS Free Tier » EMR Resizing Cluster » RDS Reserved » CloudFront Default Root » Startup Challenge 2010 » CloudFront Invalidation » CloudFront HTTPS » NYC Edge Location » Lowers Pricing HTTP » AWS Import Export GA » Amazon SNS » Amazon S3 Console » Amazon EBS CloudWatch » Amazon SNS » Combined AWS Data Transfer Savings » Amazon EMR Bootstrap Actions » Amazon ELB Session Stickiness » Amazon RDS in EU » New Singapore Region » EMR JobFlow Debugging » Simple DB Consistent Reads » Simple DB Conditional Puts » VPC in EU » Amazon RDS in US-west » Amazon CloudFront Access Logs » Amazon RDS Multi-AZ » Amazon S3 RRS » Amazon RDS Console » Amazon SQS Longer retention, Free Tier Amazon S3 Bucket Policies » Amazon VPC IP Address » Cluster Compute Instances » Amazon S3 RRS Notifications » Lowered Pricing EC2 » AWS IAM » Amazon VPC Console » Micro Instances » Amazon Linux AMI » Amazon EC2 Tagging, Filtering, Idempotency, » Oracle Certified AWS » AWS PHP SDK » Amazon S3 Lowered Pricing » CloudFront GA, SLA » S3 Multipart » GPGPU Instance Types » ISO27001/2 Certification » AWS Elastic Beanstalk » Amazon Simple Email Service » Improved AWS Support “Bronze” » Amazon CloudWatch Console » AWS CloudFormation » Amazon S3 Static Websites » AWS IAM Website Login » Paris Edge Location » VM Connector » Tokyo Region » AWS Support JP » Amazon EC2 Reserved Instances with Windows, Extra Large High Memory Instances » Amazon S3 Versioning Feature » Consolidated Billing for AWS » Lower pricing for Outbound Data Transfer » AWS Java SDK » Windows BYOL » Singapore Pop » CloudFront Private Streaming » Free Monitoring EC2 » Amazon Route 53 » PCI DSS Level 1 Certification » Mobile SDKs (Android, iPhone) » Large Object S3 Support » Florida POP » Import/Export APAC » New VPC » Dedicated Instances » Windows 2008 R2
- 7. The “Living and Evolving” AWS Cloud Low-level building blocks High-level building blocks
- 10. The “Living and Evolving” AWS Cloud Low-level building blocks High-level building blocks Tools to access services Cross Service features
- 11. Innovation continues in all dimensions
- 12. Innovative Business Models Cloud HPC: Cluster Compute Instance 2 Xeon quad-core + 2 tesla NVIDIA GPGPU + 10G non-blocking Ethernet For Spiky Workloads For Steady State Workloads For Time-insensitive workloads For Regulatory and Compliant Workloads
- 13. US West (Northern California) US East (Northern Virginia) Europe West (Dublin) Asia Pacific Region (Singapore) Ashburn, Dallas, Los Angeles, Miami, Newark, Palo Alto, Seattle, St. Louis, Amsterdam, Dublin, Frankfurt, London, Hong Kong, Singapore, Tokyo Amazon CloudFront Edge Locations Asia Pacific Region (Japan)
- 14. The “Living and Evolving” AWS Cloud Low-level building blocks High-level building blocks Tools to access services Cross Service features
- 15. #1 Flexibility and the power of choice
- 16. Can use any Programming Model, Language or Operating System or Database Can use any service individually or in any combination Can use as much or as little and only pay for what was consumed Can use any existing System Management Tools and extend the datacenter into the Cloud Flexibility is the key
- 17. The Cloud
- 18. The Cloud as a Platform
- 19. Platform that provides foundation to build innovation solutions on top Platform that provides abstraction to hide underlying layers (hardware and software) Platform that is self-service
- 20. The Cloud as a Platform
- 22. AWS Master Node Input S3 bucket Output S3 bucket Amazon S3 Input dataset output results Start From IDE Command line Web Console End Notify Get Results Input Data Amazon EC2 Instances
- 23. Wall Street & Amazon EC2 Number of EC2 Instances 300 CPU’s on weekends Thursday 4/23/2009 Friday 4/24/2009 Sunday 4/26/2009 Monday 4/27/2009 Tuesday 4/28/2009 Saturday 4/25/2009 Wednesday 4/22/2009 3000- - 300 --
- 24. 500 Hours 1 Job, 1 Machine, 500 hours 1 Job, 500 Machines, 1 hour One Hour $1500 $260 To rack and stack on-premise For 3 years (reserved 100% utilized)
- 28. #2 Elasticity is the fundamental property of the cloud
- 29. Turning off = Stop paying Infrastructure Cost $ time Large Capital Expenditure You just lost customers Predicted Demand Traditional Hardware Actual Demand Cloud Automated Elasticity
- 30. Automate Using Cloud APIs Mr. Automate Development Monitoring Management Logistics Deployment
- 31. Build a Web application that sleeps at night or shrinks itself when there are no users accessing it
- 32. Build a Web application that Follows the Sun and leverages the true power of on-demand elasticity and globally dispersed regions
- 33. US-West-1b RDS Multi-AZ US-West Auto Scaling group : Web App Tier RDS Master US East Traffi c US West Traffic ELB Geo IP/Directional DNS Server Asia Traffic DNS CNAME Software-based Data Replicator Europe Traffic US-East-1b RDS Multi-AZ US-East Auto Scaling group : Web App Tier RDS Master ELB EU-West-1b RDS Multi-AZ EU-West Auto Scaling group : Web App Tier RDS Master ELB AP-SOUTHEAST-1b RDS Multi-AZ AP-SOUTHEAST Auto Scaling group : Web App Tier RDS Master ELB Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App
- 34. US-West-1b RDS Multi-AZ US-West Auto Scaling group : Web App Tier RDS Master US East Traffi c US West Traffic ELB Geo IP/Directional DNS Server Asia Traffic DNS CNAME Software-based Data Replicator Europe Traffic US-East-1b RDS Multi-AZ US-East Auto Scaling group : Web App Tier RDS Master ELB EU-West-1b RDS Multi-AZ EU-West Auto Scaling group : Web App Tier RDS Master ELB AP-SOUTHEAST-1b RDS Multi-AZ AP-SOUTHEAST Auto Scaling group : Web App Tier RDS Master ELB Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App
- 35. Implement Elasticity in your architecture
- 36. Scalability , Security , High availability , Fault-tolerance , Testability and Elasticity will be configurable properties of the application architecture and will be an automated and intrinsic part of the platform on which they are built. The day is not too far….
- 39. 350,000 Minutes (7-8 Months) Time to provision a server in an enterprise Time to provision a server in the cloud <5 Minutes $1000 To rack and stack on-premise $260 For 3 years (reserved 100% utilized)
- 40. Amazon EC2 in 5 screens
- 51. Casual Online Games “ Gamification”
- 54. Enterprise IT
- 55. Let go of (physical) control but retain your ownership
- 56. Let go of (physical) control but retain your ownership
- 58. Identify the right candidate for the cloud Dash board
- 59. Identify the right candidate for the cloud Dash board
- 60. The New Amazon Virtual Private Cloud (VPC)
- 61. VPC is part of the Autodesk internal network
- 62. The Automation You Always Meant to Build Provision and attach 1TB of storage in 2 minutes (from the back of an auto-rickshaw in India). 10 new Linux servers in 2 minutes (while sitting by the pool on a nice day). Monitoring server resources from an iPhone (in a bar).
- 63. Pre-configured IT security-blessed auto-scalable auto-backup multi-AZ full-control one-click deployable enterprise apps App Vending Machine
- 67. Amazon S3 Momentum 2 years ago Peak Requests: 70,000 per second 200 Million 5 Billion 18 Billion 52 Billion Total Number of Objects Stored in Amazon S3 Q1 2009
- 68. The Cloud Scales: Amazon S3 Growth Peak Requests: 200,000+ per second Total Number of Objects Stored in Amazon S3 2.9 Billion 14 Billion 40 Billion 102 Billion 262 Billion
- 69. Smugmug > 2PB of Media
- 73. Recovery.gov and Treasury.gov First government-wide system to move to the cloud Savings of over $750,000 in current budget cycle “ Cloud computing strikes me as a perfect tool to help achieve greater transparency and accountability. Moving to the cloud allows us to provide better service at lower costs. I hope this development will inspire other government entities to accelerate their own efforts. The American taxpayers would be the winners.’’ - Earl E. Devaney, the Board’s Chairman.
- 76. #3 Redefining Cloud Security
- 77. In the Cloud, Security is a Shared Responsibility SAS 70 Type II Audit ISO 27001/2 Certification PCI DSS 2.0 Level 1-5 HIPAA/SOX Compliance FISMA A&A Low Enforce IAM policies Use MFA, VPC, Leverage S3 bucket policies, EC2 Security groups, EFS in EC2 Etc.. Encrypt data in transit Encrypt data at rest Protect your AWS Credentials Rotate your keys Secure your application How we secure our infrastructure What security options and features are available to you? How can you secure your application and what is your responsibility?
- 79. Build security in every layer Old World New World
- 82. The “Living and Evolving” AWS Cloud Low-level building blocks High-level building blocks Tools to access services Cross Service features
- 83. The “Living and Evolving” AWS Cloud Low-level Infrastructure building blocks High-level Infrastructure building blocks Tools to access services Cross Service features
- 84. Cloud Services for Mathematica Drupal PHP Platform Ruby/Rails Platform Mathematica Platform Voice Platform Mobile Trading Platform Geospatial Platform Location-based App Platform
- 86. Internet as the Operating System of Services
- 89. #2 Elasticity is fundamental Property of the cloud and implement elasticity #1 flexibility is very important when you chose the cloud #3 In the Cloud, Security is shared responsibility. Its has to be built in every layer #4 The Cloud is much more than its services. Its all about the Ecosystem
- 90. Thank you! Jinesh Varia jvaria@amazon.com Twitter:@jinman
Editor's Notes
- Cloud landscape is dramatically changing every day and Cloud is changing the way we do business today. How we procure hardware, How we design and architect applications, how we secure applications and how we deploy applications. Best way to learn about the cloud is to how others are using it. So I am going to discuss how the cloud is being used as a platform.
- Cloud that existed two years ago is not the same that exists today and will not be the same two years from now. This book was recently published and the reason I recommend this book is it does not give you one view of the world from one or two authors. 40 different authors contributed a total of 25 different chapters including mine and even though the book is targeted to be a research textbook for graduate students, it is a great read
- This is highly relevant in the cloud world because innovation is happening at break neck speed…..
- Make a joke…. Cannot fit in one slide…..anymore so (show next slide) 11/2, Amazaon S3 price reduction 10/21, AWS Free Usage Tier 10/20, Resizable Elastic Map Reduce jobs 10/18,Feature Release,Mgmt Console adds support for Simple Notifications Service 10/14,Feature Release,Elastic Load Balancer termination of SSL certs 10/8,Feature Release,Mgmt console support for Amazon Relational Database Service DB Engine Version 10/6,Feature Release: Support for Suse Linux 10/6,Price Reduction,RDS 10/6,Feature Release,RDS Read Replicas 9/29,Feature Release,Download invoices 9/21,Feature Release,Amazon EC2 on the Oracle Virtual Machine and full Oracle Support, Certification and License portability 9/20,Feature Release,Resource tagging & associated features 9/17,Case Study,Matlab HPC benchmark 9/15,Feature Release,Amazon Linux 9/8,Feature Release,Mgmt console support for VPC 9/2,Price reduction,M2.2x & M2.4x 8/31,Case study,Authority to Operate-Appian BPM 8/25,General Announcement,Updated Security Whitepaper 8/24,Feature Release,Java SDK updated 8/17,Feature Release,RDS reserved instances 8/17,Feature Release,MySQL DB engine version mgmt 8/6,Feature Release,CloudFront Default Root Object 7/23,Opperational Change,Vulnerability Reporting and Pen Testing 7/16,Feature Release,Use your own kernel 7/15,Feature Release,Enhanced CF logs w/ Query strings 7/14,Feature Release,RRS in Mgmt Console 7/14,Feature Release,VPC IP address control & config file generation 6/29,Feature Release,RDS support of SSL
- Amazon only offers infrastructure as a service.
- The cloud is living and continuously evolving. AWS not only offers low-level services but also higher-level services with full flexibility
- Innovation is not just in technology but also in business models. 4 new ways to purchase an EC2 instance. Imagine when you can combine these with Cloud HPC instances. It gives business the power to do pretty Extraordinary things.
- And more expansion and more geographies
- First Take away
- AWS does not force developers to use any particular programming model, language, or operating system. AWS does not force developers to use the entire suite of services - they can use any of our infrastructure services individually or in any combination. AWS does not limit developers to a pre-set amount of storage, bandwidth, or computing resources they can consume - they can use as much or as little as they wish, and only pay for what they use.
- This is a real usage graph from one of our financial services customers during the last week of April (They have asked to remain anonymous for competitive reasons). Firms on Wall Street are finding EC2 an ideal environment to run many of their daily mission critical grid computing and cpu bound applications for a couple key reasons: 1/ Flexibilitythe ability to instantly access hundreds/thousands of cores increases the amount of data they can process, improving the overall quality of their models. and 2/ Cost efficiencies, as they can complete more of their processing for less total spend (Not paying for infrastructure during times of the day and weekends when its not needed) This wall street firm in particular has a nightly business process where they upload the day’s market trading data into S3, and then run proprietary ‘risk management’ algorithms. This lasts ~10 hours during week nights, where they ramp up to the equivalent 3000 m1.smalls. During the day and on weekends, they maintain a base of roughly 300 cores, to handle their always on work loads.
- This slide applies to Amazon EC2, but just as easily describes Amazon S3’s value proposition.
- The day is not too far when applications will cease to be aware of physical hardware. Much like plugging in a microwave in order to power it doesn’t require any knowledge of electricity, one should be able to plug in an application to the cloud in order to receive the power it needs to run, just like a utility. As an architect, you will manage abstract compute, storage and network resources instead of physical servers. Applications will continue to function even if the underlying physical hardware fails or is removed or replaced. Applications will adapt themselves to fluctuating demand patterns by deploying resources instantaneously and automatically, thereby achieving highest utilization levels at all times. Scalability, Security, High availability, Fault-tolerance, Testability and Elasticity will be configurable properties of the application architecture and will be an automated and intrinsic part of the platform on which they are built. However, we are not there yet. Today, you can build applications in the cloud with some of these qualities by implementing the best practices highlighted in the paper. Best practices in cloud computing architectures will continue to evolve and as researchers, we should focus not only on enhancing the cloud but also on building tools, technologies and processes that will make it easier for developers and architects to plug in applications to the cloud easily.
- In eclipsecon 2010 – a conference focussed on eclipse. They launched a contest for developers to build apps that will drive the lego mindstorm robot using APIs. There were 4X4 arena which collected and stored images in the cloud that shows the location
- What took 15 days to process, can now be processed in 2 hours
- These games architectures need scaling like we have never seen before. They need cloud
- Remember TV’s without remote control. We had to walk up to the television set to change the channel or the volume. Now we have remote controls, we sit on our couch and control the hardware. Cloud APIs are remote control of the cloud hardware. You don’t have the need to walk up to server hardware.
- Remember TV’s without remote control. We had to walk up to the television set to change the channel or the volume. Now we have remote controls, we sit on our couch and control the hardware. Cloud APIs are remote control of the cloud hardware. You don’t have the need to walk up to server hardware.
- Point of Slide: to explain VPC's high-level architecture, walking them through the discrete elements of a VPC, and a specific data flow to exemplify 1) data-in-transit security and continued 1) AAA control by the enterprise. AWS (”orange cloud&quot;): What everybody knows of AWS today. Customer’s Network (“blue square”) : The customer’s internal IT infrastructure. VPC (”blue square on top of orange cloud&quot;): Secure container for other object types; includes Border Router for external connectivity. The isolated resources that customers have in the AWS cloud. Cloud Router (“orange router surrounded by clouds”) : Lives within a VPC; anchors an AZ; presents stateful filtering. Cloud Subnet (“blue squares” inside VPC): connects instances to a Cloud Router. VPN Connection: Customer Gateway and VPN Gateway anchor both sides of the VPN Connection, and enables secure connectivity; implemented using industry standard mechanisms. Please note that we currently require whatever customer gateway device is used supports BGP. We actually terminate two (2) tunnels - one tunnel per VPN Gateway - on our side. Besides providing high availability, we can service one device while maintaining service. As such, we can either connect to one of the customer's BGP-supporting devices (preferably running JunOS or IOS).
- From sound-proof room to noise-cancelling headphones – it gives you flexibility to move Traditional Perimeter security to securing individual applications
- 4 Key take aways!