• Save
The Cloud as a Platform - Cloud Connections 2011 Keynote - Jinesh Varia
Upcoming SlideShare
Loading in...5
×
 

The Cloud as a Platform - Cloud Connections 2011 Keynote - Jinesh Varia

on

  • 4,164 views

The Cloud as a Platform Keynote Presentation delivered at Cloud Connections Conference (DevConnections) April 19, 2011 by Jinesh Varia, Technology Evangelist, Amazon

The Cloud as a Platform Keynote Presentation delivered at Cloud Connections Conference (DevConnections) April 19, 2011 by Jinesh Varia, Technology Evangelist, Amazon

Statistics

Views

Total Views
4,164
Views on SlideShare
4,014
Embed Views
150

Actions

Likes
23
Downloads
0
Comments
0

3 Embeds 150

http://www.windowsitpro.com 93
http://www.clouditproonline.com 54
http://windowsitpro.com 3

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Cloud landscape is dramatically changing every day and Cloud is changing the way we do business today. How we procure hardware, How we design and architect applications, how we secure applications and how we deploy applications. Best way to learn about the cloud is to how others are using it. So I am going to discuss how the cloud is being used as a platform.
  • Cloud that existed two years ago is not the same that exists today and will not be the same two years from now. This book was recently published and the reason I recommend this book is it does not give you one view of the world from one or two authors. 40 different authors contributed a total of 25 different chapters including mine and even though the book is targeted to be a research textbook for graduate students, it is a great read
  • This is highly relevant in the cloud world because innovation is happening at break neck speed…..
  • Make a joke…. Cannot fit in one slide…..anymore so (show next slide) 11/2, Amazaon S3 price reduction 10/21, AWS Free Usage Tier 10/20, Resizable Elastic Map Reduce jobs 10/18,Feature Release,Mgmt Console adds support for Simple Notifications Service 10/14,Feature Release,Elastic Load Balancer termination of SSL certs 10/8,Feature Release,Mgmt console support for Amazon Relational Database Service DB Engine Version 10/6,Feature Release: Support for Suse Linux 10/6,Price Reduction,RDS 10/6,Feature Release,RDS Read Replicas 9/29,Feature Release,Download invoices 9/21,Feature Release,Amazon EC2 on the Oracle Virtual Machine and full Oracle Support, Certification and License portability 9/20,Feature Release,Resource tagging & associated features 9/17,Case Study,Matlab HPC benchmark 9/15,Feature Release,Amazon Linux 9/8,Feature Release,Mgmt console support for VPC 9/2,Price reduction,M2.2x & M2.4x 8/31,Case study,Authority to Operate-Appian BPM 8/25,General Announcement,Updated Security Whitepaper 8/24,Feature Release,Java SDK updated 8/17,Feature Release,RDS reserved instances 8/17,Feature Release,MySQL DB engine version mgmt 8/6,Feature Release,CloudFront Default Root Object 7/23,Opperational Change,Vulnerability Reporting and Pen Testing 7/16,Feature Release,Use your own kernel 7/15,Feature Release,Enhanced CF logs w/ Query strings 7/14,Feature Release,RRS in Mgmt Console 7/14,Feature Release,VPC IP address control & config file generation 6/29,Feature Release,RDS support of SSL
  • Amazon only offers infrastructure as a service.
  • The cloud is living and continuously evolving. AWS not only offers low-level services but also higher-level services with full flexibility
  • Innovation is not just in technology but also in business models. 4 new ways to purchase an EC2 instance. Imagine when you can combine these with Cloud HPC instances. It gives business the power to do pretty Extraordinary things.
  • And more expansion and more geographies
  • First Take away
  • AWS does not force developers to use any particular programming model, language, or operating system. AWS does not force developers to use the entire suite of services - they can use any of our infrastructure services individually or in any combination. AWS does not limit developers to a pre-set amount of storage, bandwidth, or computing resources they can consume - they can use as much or as little as they wish, and only pay for what they use.
  • This is a real usage graph from one of our financial services customers during the last week of April (They have asked to remain anonymous for competitive reasons). Firms on Wall Street are finding EC2 an ideal environment to run many of their daily mission critical grid computing and cpu bound applications for a couple key reasons: 1/ Flexibilitythe ability to instantly access hundreds/thousands of cores increases the amount of data they can process, improving the overall quality of their models. and 2/ Cost efficiencies, as they can complete more of their processing for less total spend (Not paying for infrastructure during times of the day and weekends when its not needed) This wall street firm in particular has a nightly business process where they upload the day’s market trading data into S3, and then run proprietary ‘risk management’ algorithms. This lasts ~10 hours during week nights, where they ramp up to the equivalent 3000 m1.smalls. During the day and on weekends, they maintain a base of roughly 300 cores, to handle their always on work loads.
  • This slide applies to Amazon EC2, but just as easily describes Amazon S3’s value proposition.
  • The day is not too far when applications will cease to be aware of physical hardware. Much like plugging in a microwave in order to power it doesn’t require any knowledge of electricity, one should be able to plug in an application to the cloud in order to receive the power it needs to run, just like a utility. As an architect, you will manage abstract compute, storage and network resources instead of physical servers. Applications will continue to function even if the underlying physical hardware fails or is removed or replaced. Applications will adapt themselves to fluctuating demand patterns by deploying resources instantaneously and automatically, thereby achieving highest utilization levels at all times. Scalability, Security, High availability, Fault-tolerance, Testability and Elasticity will be configurable properties of the application architecture and will be an automated and intrinsic part of the platform on which they are built. However, we are not there yet. Today, you can build applications in the cloud with some of these qualities by implementing the best practices highlighted in the paper. Best practices in cloud computing architectures will continue to evolve and as researchers, we should focus not only on enhancing the cloud but also on building tools, technologies and processes that will make it easier for developers and architects to plug in applications to the cloud easily.
  • In eclipsecon 2010 – a conference focussed on eclipse. They launched a contest for developers to build apps that will drive the lego mindstorm robot using APIs. There were 4X4 arena which collected and stored images in the cloud that shows the location
  • What took 15 days to process, can now be processed in 2 hours
  • These games architectures need scaling like we have never seen before. They need cloud
  • Remember TV’s without remote control. We had to walk up to the television set to change the channel or the volume. Now we have remote controls, we sit on our couch and control the hardware. Cloud APIs are remote control of the cloud hardware. You don’t have the need to walk up to server hardware.
  • Remember TV’s without remote control. We had to walk up to the television set to change the channel or the volume. Now we have remote controls, we sit on our couch and control the hardware. Cloud APIs are remote control of the cloud hardware. You don’t have the need to walk up to server hardware.
  • Point of Slide: to explain VPC's high-level architecture, walking them through the discrete elements of a VPC, and a specific data flow to exemplify 1) data-in-transit security and continued 1) AAA control by the enterprise. AWS (”orange cloud"): What everybody knows of AWS today. Customer’s Network (“blue square”) : The customer’s internal IT infrastructure. VPC (”blue square on top of orange cloud"): Secure container for other object types; includes Border Router for external connectivity. The isolated resources that customers have in the AWS cloud. Cloud Router (“orange router surrounded by clouds”) : Lives within a VPC; anchors an AZ; presents stateful filtering. Cloud Subnet (“blue squares” inside VPC): connects instances to a Cloud Router. VPN Connection: Customer Gateway and VPN Gateway anchor both sides of the VPN Connection, and enables secure connectivity; implemented using industry standard mechanisms. Please note that we currently require whatever customer gateway device is used supports BGP. We actually terminate two (2) tunnels - one tunnel per VPN Gateway - on our side. Besides providing high availability, we can service one device while maintaining service. As such, we can either connect to one of the customer's BGP-supporting devices (preferably running JunOS or IOS).
  • From sound-proof room to noise-cancelling headphones – it gives you flexibility to move Traditional Perimeter security to securing individual applications
  • 4 Key take aways!

The Cloud as a Platform - Cloud Connections 2011 Keynote - Jinesh Varia The Cloud as a Platform - Cloud Connections 2011 Keynote - Jinesh Varia Presentation Transcript

  •  
  • Published by Wiley (March 29, 2011) 25 Chapters contributed by various authors around the world Reference guide for Practitioners, new researchers, developers architects, and Graduate Students (textbook) Captures the state of the art in cloud computing technologies and applications
  •  
  • The pace of innovation in 2009 » Amazon RDS » High-Memory Instances » Lower EC2 Pricing » AWS Multi-Factor Authentication » Virtual Private Cloud » Lower Reserved Instance Pricing » AWS Security Center » Reserved Instances in EU Region » Elastic MapReduce » SQS in EU Region » New SimpleDB Features » FPS General Availability » Lower pricing tiers for Amazon CloudFront » AWS Management Console » Amazon EC2 with Windows » Amazon EC2 in EU Region » AWS Toolkit for Eclipse » Amazon EC2 Reserved Instances » AWS Import/Export » New CloudFront Feature » Monitoring, Auto Scaling & Elastic Load Balancing » Amazon Elastic MapReduce in Europe » EBS Shared Snapshots » SimpleDB in EU Region » Monitoring, Auto Scaling & Elastic Load Balancing in EU » Amazon CloudFront Private Content » SAS70 Type II Audit » AWS SDK for .NET
    • » Amazon EC2 with Windows Server
    • 2008,
    • Spot Instances,
    • Boot from Amazon EBS
    • » Amazon CloudFront Streaming
    • » Amazon VPC enters Unlimited Beta
    • » AWS Region in Northern California
    • » International Support for AWS
    • Import/Export
    » Amazon EC2 Reserved Instances with Windows, Extra Large High Memory Instances » Amazon S3 Versioning Feature » Consolidated Billing for AWS » Lower pricing for Outbound Data Transfer » Amazon SNS
  • And pace accelerates in 2010…. » Amazon RDS Read Replicas » Suse EC2 Linux » Amazon SNS Console » Amazon ELB HTTPS » AWS Free Tier » EMR Resizing Cluster » RDS Reserved » CloudFront Default Root » Startup Challenge 2010 » CloudFront Invalidation » CloudFront HTTPS » NYC Edge Location » Lowers Pricing HTTP » AWS Import Export GA » Amazon SNS » Amazon S3 Console » Amazon EBS CloudWatch » Amazon SNS » Combined AWS Data Transfer Savings » Amazon EMR Bootstrap Actions » Amazon ELB Session Stickiness » Amazon RDS in EU » New Singapore Region » EMR JobFlow Debugging » Simple DB Consistent Reads » Simple DB Conditional Puts » VPC in EU » Amazon RDS in US-west » Amazon CloudFront Access Logs » Amazon RDS Multi-AZ » Amazon S3 RRS » Amazon RDS Console » Amazon SQS Longer retention, Free Tier Amazon S3 Bucket Policies » Amazon VPC IP Address » Cluster Compute Instances » Amazon S3 RRS Notifications » Lowered Pricing EC2 » AWS IAM » Amazon VPC Console » Micro Instances » Amazon Linux AMI » Amazon EC2 Tagging, Filtering, Idempotency, » Oracle Certified AWS » AWS PHP SDK » Amazon S3 Lowered Pricing » CloudFront GA, SLA » S3 Multipart » GPGPU Instance Types » ISO27001/2 Certification » AWS Elastic Beanstalk » Amazon Simple Email Service » Improved AWS Support “Bronze” » Amazon CloudWatch Console » AWS CloudFormation » Amazon S3 Static Websites » AWS IAM Website Login » Paris Edge Location » VM Connector » Tokyo Region » AWS Support JP » Amazon EC2 Reserved Instances with Windows, Extra Large High Memory Instances » Amazon S3 Versioning Feature » Consolidated Billing for AWS » Lower pricing for Outbound Data Transfer » AWS Java SDK » Windows BYOL » Singapore Pop » CloudFront Private Streaming » Free Monitoring EC2 » Amazon Route 53 » PCI DSS Level 1 Certification » Mobile SDKs (Android, iPhone) » Large Object S3 Support » Florida POP » Import/Export APAC » New VPC » Dedicated Instances » Windows 2008 R2
  •  
  • The “Living and Evolving” AWS Cloud Low-level building blocks High-level building blocks
  •  
  • AWS Elastic Beanstalk
    • Deploy Versions
    • Edit Configuration
    • Monitor Health
    • View Logs
    • Restart App Server
    • Rebuild Infrastructure
    • Terminate
    Availability Zone #1 Auto Scaling Group Myapp.elasticbeanstalk.com LB Buckets 8GB Root Volume 2GB Data Volume Snapshots Logs Amazon S3 Elastic Load Balancer Amazon S3 Bucket Route53 HostedZone Amazon EC2 Micro Instance Web Server App Server
  • The “Living and Evolving” AWS Cloud Low-level building blocks High-level building blocks Tools to access services Cross Service features
  • Innovation continues in all dimensions
  • Innovative Business Models Cloud HPC: Cluster Compute Instance 2 Xeon quad-core + 2 tesla NVIDIA GPGPU + 10G non-blocking Ethernet For Spiky Workloads For Steady State Workloads For Time-insensitive workloads For Regulatory and Compliant Workloads
  • US West (Northern California) US East (Northern Virginia) Europe West (Dublin) Asia Pacific Region (Singapore) Ashburn, Dallas, Los Angeles, Miami, Newark, Palo Alto, Seattle, St. Louis, Amsterdam, Dublin, Frankfurt, London, Hong Kong, Singapore, Tokyo Amazon CloudFront Edge Locations Asia Pacific Region (Japan)
  • The “Living and Evolving” AWS Cloud Low-level building blocks High-level building blocks Tools to access services Cross Service features
  • #1 Flexibility and the power of choice
  • Can use any Programming Model, Language or Operating System or Database Can use any service individually or in any combination Can use as much or as little and only pay for what was consumed Can use any existing System Management Tools and extend the datacenter into the Cloud Flexibility is the key
  • The Cloud
  • The Cloud as a Platform
  • Platform that provides foundation to build innovation solutions on top Platform that provides abstraction to hide underlying layers (hardware and software) Platform that is self-service
  • The Cloud as a Platform
    • The Cloud as a Platform
        • For Computation
  • AWS Master Node Input S3 bucket Output S3 bucket Amazon S3 Input dataset output results Start From IDE Command line Web Console End Notify Get Results Input Data Amazon EC2 Instances
  • Wall Street & Amazon EC2 Number of EC2 Instances 300 CPU’s on weekends Thursday 4/23/2009 Friday 4/24/2009 Sunday 4/26/2009 Monday 4/27/2009 Tuesday 4/28/2009 Saturday 4/25/2009 Wednesday 4/22/2009 3000- - 300 --
  • 500 Hours 1 Job, 1 Machine, 500 hours 1 Job, 500 Machines, 1 hour One Hour $1500 $260 To rack and stack on-premise For 3 years (reserved 100% utilized)
  • “ TimesMachine” from NYTimes
    • 1851-1922 Articles
    • TIFF -> PDF
    • Input: 11 Million Articles (4TB of data)
    • What did he do ?
      • 100 EC2 Instances for 24 hours
      • Input: All data on S3
      • Output: 1.5 TB of Data
      • Hadoop, iText, JetS3t
  •  
  • Source: http://qcontokyo.com/pdf/adriancockroft.pdf
  • #2 Elasticity is the fundamental property of the cloud
  • Turning off = Stop paying Infrastructure Cost $ time Large Capital Expenditure You just lost customers Predicted Demand Traditional Hardware Actual Demand Cloud Automated Elasticity
  • Automate Using Cloud APIs Mr. Automate Development Monitoring Management Logistics Deployment
  • Build a Web application that sleeps at night or shrinks itself when there are no users accessing it
  • Build a Web application that Follows the Sun and leverages the true power of on-demand elasticity and globally dispersed regions
  • US-West-1b RDS Multi-AZ US-West Auto Scaling group : Web App Tier RDS Master US East Traffi c US West Traffic ELB Geo IP/Directional DNS Server Asia Traffic DNS CNAME Software-based Data Replicator Europe Traffic US-East-1b RDS Multi-AZ US-East Auto Scaling group : Web App Tier RDS Master ELB EU-West-1b RDS Multi-AZ EU-West Auto Scaling group : Web App Tier RDS Master ELB AP-SOUTHEAST-1b RDS Multi-AZ AP-SOUTHEAST Auto Scaling group : Web App Tier RDS Master ELB Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App
  • US-West-1b RDS Multi-AZ US-West Auto Scaling group : Web App Tier RDS Master US East Traffi c US West Traffic ELB Geo IP/Directional DNS Server Asia Traffic DNS CNAME Software-based Data Replicator Europe Traffic US-East-1b RDS Multi-AZ US-East Auto Scaling group : Web App Tier RDS Master ELB EU-West-1b RDS Multi-AZ EU-West Auto Scaling group : Web App Tier RDS Master ELB AP-SOUTHEAST-1b RDS Multi-AZ AP-SOUTHEAST Auto Scaling group : Web App Tier RDS Master ELB Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App Web App
  • Implement Elasticity in your architecture
  • Scalability , Security , High availability , Fault-tolerance , Testability and Elasticity will be configurable properties of the application architecture and will be an automated and intrinsic part of the platform on which they are built. The day is not too far….
    • The Cloud as a Platform
        • For Computation
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
  • 350,000 Minutes (7-8 Months) Time to provision a server in an enterprise Time to provision a server in the cloud <5 Minutes $1000 To rack and stack on-premise $260 For 3 years (reserved 100% utilized)
  • Amazon EC2 in 5 screens
  •  
  •  
  •  
  •  
  •  
  •  
  • Amazon Web Services Availability Zone 2
    • Amazon SimpleDB
    • Domains
    • Registration
    • Scores
    • -Commands
    • -Game Queue
    • -Logging
    Amazon S3 Availability Zone 1 Auto-scaling group API Endpoint ELB Equinox Server Equinox Server EBS EBS RESTFul API RESTFul API Conditional Puts Consistent Reads Version Aware Lookups 4images Every Sec From Camera E4 Clients (Mac, Linux, Windows UI, iPhone, JoySticks ) CloudWatch MindStorm Robot controlled by Arena Server Arena Server Open Social Gadgets Showing images almost in real time Polls for commands Sends commands
  •  
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
        • For New Generation Apps
  • Casual Online Games “ Gamification”
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
        • For New Generation Apps
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
        • For New Generation Apps
        • For Internal Corporate IT
  • Enterprise IT
  • Let go of (physical) control but retain your ownership
  • Let go of (physical) control but retain your ownership
  • Create a Dependency Tree of your IT Assets
    • List all your IT assets
    • Identify upward and downward dependencies
    • Start classifying your IT assets into different categories:
      • Applications with Top Secret, Secret, or Public data sets
      • Applications with low, medium and high compliance requirements
      • Applications that are internal-only, partner-only or customer-facing
      • Applications with low, medium and high coupling
      • Applications with strict, relaxed licensing
    Dash board
  • Identify the right candidate for the cloud Dash board
  • Identify the right candidate for the cloud Dash board
  • The New Amazon Virtual Private Cloud (VPC)
  • VPC is part of the Autodesk internal network
  • The Automation You Always Meant to Build Provision and attach 1TB of storage in 2 minutes (from the back of an auto-rickshaw in India). 10 new Linux servers in 2 minutes (while sitting by the pool on a nice day). Monitoring server resources from an iPhone (in a bar).
  • Pre-configured IT security-blessed auto-scalable auto-backup multi-AZ full-control one-click deployable enterprise apps App Vending Machine
  •  
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
        • For New Generation Apps
        • For Internal Corporate IT
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
        • For New Generation Apps
        • For Internal Corporate IT
        • For Storage
  • Amazon S3 Momentum 2 years ago Peak Requests: 70,000 per second 200 Million 5 Billion 18 Billion 52 Billion Total Number of Objects Stored in Amazon S3 Q1 2009
  • The Cloud Scales: Amazon S3 Growth Peak Requests: 200,000+ per second Total Number of Objects Stored in Amazon S3 2.9 Billion 14 Billion 40 Billion 102 Billion 262 Billion
  • Smugmug > 2PB of Media
  •  
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
        • For New Generation Apps
        • For Internal Corporate IT
        • For Storage
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
        • For New Generation Apps
        • For Internal Corporate IT
        • For Storage
        • For Integration
  • Recovery.gov and Treasury.gov First government-wide system to move to the cloud Savings of over $750,000 in current budget cycle “ Cloud computing strikes me as a perfect tool to help achieve greater transparency and accountability. Moving to the cloud allows us to provide better service at lower costs. I hope this development will inspire other government entities to accelerate their own efforts. The American taxpayers would be the winners.’’ - Earl E. Devaney, the Board’s Chairman.
  • Recovery.gov Cloud Architecture
  •  
  • #3 Redefining Cloud Security
  • In the Cloud, Security is a Shared Responsibility SAS 70 Type II Audit ISO 27001/2 Certification PCI DSS 2.0 Level 1-5 HIPAA/SOX Compliance FISMA A&A Low Enforce IAM policies Use MFA, VPC, Leverage S3 bucket policies, EC2 Security groups, EFS in EC2 Etc.. Encrypt data in transit Encrypt data at rest Protect your AWS Credentials Rotate your keys Secure your application How we secure our infrastructure What security options and features are available to you? How can you secure your application and what is your responsibility?
  •  
  • Build security in every layer Old World New World
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
        • For New Generation Apps
        • For Internal Corporate IT
        • For Storage
        • For Integration
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
        • For New Generation Apps
        • For Internal Corporate IT
        • For Storage
        • For Integration
        • For Platforms
  • The “Living and Evolving” AWS Cloud Low-level building blocks High-level building blocks Tools to access services Cross Service features
  • The “Living and Evolving” AWS Cloud Low-level Infrastructure building blocks High-level Infrastructure building blocks Tools to access services Cross Service features
  • Cloud Services for Mathematica Drupal PHP Platform Ruby/Rails Platform Mathematica Platform Voice Platform Mobile Trading Platform Geospatial Platform Location-based App Platform
  •  
  • Internet as the Operating System of Services
    • The Cloud as a Platform
        • For Computation
        • For Research and Development
        • For New Generation Apps
        • For Internal Corporate IT
        • For Storage
        • For Collaboration
        • For Platforms
    • The Cloud as a Platform
        • For Realizing your ideas
  • #2 Elasticity is fundamental Property of the cloud and implement elasticity #1 flexibility is very important when you chose the cloud #3 In the Cloud, Security is shared responsibility. Its has to be built in every layer #4 The Cloud is much more than its services. Its all about the Ecosystem
  • Thank you! Jinesh Varia jvaria@amazon.com Twitter:@jinman
  • http://aws.amazon.com