A Practice of Cloud Computing
Upcoming SlideShare
Loading in...5
×
 

A Practice of Cloud Computing

on

  • 841 views

 

Statistics

Views

Total Views
841
Views on SlideShare
840
Embed Views
1

Actions

Likes
0
Downloads
18
Comments
0

1 Embed 1

http://www.slideshare.net 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Key Points: By leveraging the power of Amazon Web Services, the NY Times was able to utilize hundreds of machines concurrently and process all the data in less than 36 hours. Efficiency exemplified! No captial expense for equipment Ability to deploy in hours not months http://open.blogs.nytimes.com/2008/05/21/the-new-york-times-archives-amazon-web-services-timesmachine/ http://aws.typepad.com/aws/2008/05/new-york-times.html http://qik.com/video/82178
  • 年複合成長率  CAGR 。 Compound Annual Growth Rate 。 舉個實例來說明會比較容易了解:若甲產品 2001 年預估銷售值為 1956 元, 2003 年時可達 2263 元,這兩年的產值成長 15.69% 〔 2263/1956 * 100 〕,則 (1+X%) 的平方為 1.1569 , X 就等於 1.1569 開平方根再減 1 ,可得到 7.6% ,因此,甲產品的年複合成長率為 7.6% 。
  • Key points: Cloud is a popular current buzzword. Start-ups have cloud terminology in their business plans. All the big players in IT have a cloud program. Pretty much anything that translates to paying less for today's infrastructure is being rolled up under the banner of cloud computing. But cloud means different things to different people. Details: There are two things driving the excitement around cloud computing: 1) Controlling costs – amplified by today's economy, there is a need to get much more efficient in building next-generation data centers 2) An explosion in unstructured data – not just what we put into Oracle or MySQL, but click data, behavior data, rich data, tag data. Everything from the terabyte of pictures that go onto Facebook every day to the clickstream data on a wireless network from people who are consuming content on their phones. Unstructured data is forcing us to look at new ways of managing and deriving value from the data. Transition: Cloud computing brings a new level of efficiency to delivering IT resources on demand; and in the process it opens up new business models and market opportunities.
  • Another dimension is that there will be many clouds, provided by different service providers, to meet different business needs.  We can think about this in terms of different application areas. Web : Wants the best throughput at the lowest cost HPC and Analytics : Wants maximum performance. Fewer requirements around security and privacy. More around scale and elasticity. Regulated Applications : Medical and financial applications will come with a set of requirements security and privacy requirements – HPPA. SEC requirements, etc.   This leads to a range of clouds that will exist in the marketplace and Sun is looking to be a player in all of these.
  • The simplest way to look at the various layers of cloud computing is through the following three categories. 1) Infrastructure as a Service – or as some of the industry analysts are starting to refer to it, Hardware as a Service. Basic storage and compute, offered up on the network as something under programmatic provisioning and control. Amazon's EC2 and S3, Microsoft's Cloud Infrastructure Services, Rackspace's Mosso offering. Gives you the ability to provision a certain number of machine images, and a certain amount of storage, for a certain amount of time, and when you're done using them, you stop paying for them. There are very few constraints on what developers do at this layer. 2) Platform as a Service – more of a developer-targeted offering, a set of built-in services that provide a starting point for developers to deploy to. Google's App Engine, Microsoft's Azure. However, each one of these comes with a set of constraints. With Microsoft, you're writing in .Net. With App Engine, you're using Big Table and you're writing in Python. If you want to deploy C++ code on top of Oracle, you can't do it on App Engine. You may be able to build a machine image that uses those elements, and take it to an IaaS provider. But you're probably not going to do it at the platform layer. 3) Software as a Service – software, offered on the network, as a service. We've been talking about this in the industry for a long time. Applications on demand, like salesforce.com, or Google Docs online, where essentially what you're worrying about is the user interface and SLA. [NOTE: What's important here is to identify the layer of abstraction our customers want to get to. Is it to run anything in the cloud? To run a developer-specific set of services? Or to go consume an end point? An ISV will be most interested in how they become a SaaS provider. A customer in the cloud on-ramp space (load balancing, for example) may be interested in how they become a PaaS provider. And a lot of our enterprise customers, and SMBs are interested in simply consuming IaaS.]
  • Key points: While they share common elements, there are a variety of ways of looking at clouds. [NOTE: This is a great starting point for a customer discussion. How do they see their enterprise or their business using clouds?] Details: What are the different layers at which we might provide cloud services? What are the business models under which clouds will be operated and used? What are the different types of applications you want to put in to the cloud? Transition: Let's look at each of these elements in more detail.
  • UniCluster introduces the concept of ‘Kits’. A Kit is a meta-rpm package for installing software into a cluster. As you can see from the diagram Kits may contain more than one component. Special kit and component rpms define plugins for configuring pre and post installation scripts including all the dependencies. Components are individual ‘units’ of software that can be installed on a node and perform a complete function – for example in the SGE kit there is a SGE compute node component that installs the SGE execution daemon onto the nodes. Kits are fully integrated into yum and rpm thus dependencies are automatically resolved for kits when they are installed into the cluster.
  • Some I/O standards do support IO virtualization like PCI-Express, Infiniband, etc. Networking Isolation: Cisco Nexus 1000V Series Switches, an intelligent software switch implementation for Vmware vSphere environment, Operating inside the VMware ESX hypervisor, the Cisco Nexus 1000V Series supports Cisco VN-Link server virtualization technology to provide: • Policy-based virtual machine connectivity • Mobile virtual machine security and network policy • Nondisruptive operational model for your server virtualization, and networking teams Open vSwitch is a multilayer virtual switch licensed under the open source Apache 2 license.  Our goal is to build a production quality switch for VM environments that supports standard management interfaces (e.g. NetFlow, RSPAN, ERSPAN, IOS-like CLI), and is open to programmatic extension and control.  In addition, it is designed to support distribution across multiple physical servers similar to VMWare’s distributed vswitch or Cisco’s Nexus 1000v. Open vSwitch currently supports multiple virtualization technologies. It is the default switch in the Xen Cloud Paltform . And it supports open source Xen, XenServer, KVM, and VirtualBox. The bulk of the code is written in platform-independent C and is easily ported to other environments. Cisco Nexus v1000 Switches: http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9902/guide_c07-556626.html Open vSwitch: http://openvswitch.org/
  • Key points: While they share common elements, there are a variety of ways of looking at clouds. [NOTE: This is a great starting point for a customer discussion. How do they see their enterprise or their business using clouds?] Details: What are the different layers at which we might provide cloud services? What are the business models under which clouds will be operated and used? What are the different types of applications you want to put in to the cloud? Transition: Let's look at each of these elements in more detail.

A Practice of Cloud Computing A Practice of Cloud Computing Presentation Transcript

  • A Practice of Cloud Computing for HPC & Other Applications Matthew Huang Sun Microsystems, a subsidiary of Oracle Corp. [email_address]
    • IT Transformation to Cloud Computing
  • Example: NY Times TimesMachine
    • Massive data archives
      • Every newspaper from 1851 to 1922
      • http://timesmachine.nytimes.com
    • Utilizes Amazon Web Services (public cloud) and Hadoop (OpenSolaris)
    • 405,000 very large TIFF images, 3.3 million articles in SGML and 405,000 xml files -> converted to a more web-friendly 810,000 PNG images and 405,000 JavaScript files
    • Created in less than 36 hours by utilizing hundreds of machines
  • US White House Wants Cloud
    • Page 157 of US Cross Cutting Document: http://www.whitehouse.gov/omb/budget/fy2010/assets/crosscutting.pdf
    • Optimizing Common Services and Solutions
      • Cloud-Computing Platform—The Federal technology environment requires a fundamental reexamination of investments in technology infrastructure. The Infrastructure Modernization Program will be taking on new challenges and responsibilities.
    • GSA.gov already in cloud.
    • U.S. Federal Cloud Computing Market Forecast 2010 – 2015 http://www.marketresearchmedia.com/2009/05/20/us-federal-cloud-computing-market-forecast-2010-2015/
    • ( 註 )CAGR= 年複和 成長率
  • Cloud Types and Cloud Development 1. 1. 2. 2. 雲的種類 雲的形成要素
  • IT Transformation into cloud
  • Everyone is Talking About Clouds Database as a Service Utility Computing Virtualization Application Hosting Infrastructure as a Service Grid Computing Platform as a Service Storage as a Service Software as a Service Utility Computing
  • Faster time-to-market Reduction of custom software Pay only for what you use Grow infrastructure with business Faster time-to-market Reduction of custom software Pay only for what you use Grow infrastructure with business Domains Drive Differences in Hardware and Software Architecture HPC Medical Intelligence Finance Analytics Web
  • 5 Principal Characteristics of Cloud Computing
    • Abstraction of Infrastructure
      • Virtualization at the hyper-visor, OS or higher levels customized file system, OS or communication protocols.
    • Resource Democratization
      • Portability & Mobility between clouds is possible.
      • Root if required
    • Services Oriented Architecture
      • Access loosely-coupled resources in a standard way. The focus is on the delivery of service and not the management of infrastructure.
    • Elasticity/Dynamism of Resources
      • Scale in minutes, Load mgmt & balancing within pooled resources.
    • Utility model of Consumption & Allocation
      • All-you-can-eat but pay-by-the-bite (pay-per-use) pricing model
  • Perspectives of the Cloud - Summary
  • Faster time-to-market Reduction of custom software Pay only for what you use Grow infrastructure with business Faster time-to-market Reduction of custom software Pay only for what you use Grow infrastructure with business Applications offered on-demand over the network (Salesforce.com, Oracle CRM on-demand) Basic storage and compute capabilities offered as a service (Amazon web services) Developer platform with built-in services (Google App Engine) Infrastructure as a Service Platform as a Service Software as a Service
  • Cloud Computing Road To Business Value Industrialisation Virtualization Automation Infrastructure Maturity Business Value Standardization Consolidation Governance Infrastructure Maturity Business Value Performance Management Infrastructure virtualization End-to-end services provisioning Infrastructure on demand Cloud Computing Hig Performance Computing Step #1 Step #2 Step #3 Level 2 IT Component Management Level 1 Crisis Control Level 5 Business Value Management Level 4 IT Service Management Level 3 IT Operation Management
  • A Practice of Cloud Computing for HPC
  • Solutions built on Oracle Technology by
    • Oracle Enterprise Linux
      • Binary compatibility – viable alternative
      • More for less
    • Oracle VM
      • Fast!! It’s really fast…
      • Leverage Oracle VM Management
    • Oracle DB
      • High throughput reporting
    • Oracle e-Business Suite
      • PaaS and SaaS is a large part of our GTM
      • EBS market share
      • Break the silo – share
    • Sun Grid Engine
      • The best open-sourced DRM for HPC
  • Univa UD: Company Snapshot
    • The Leaders in Dynamic IT Enablement
      • Leading Provider of Cloud Management Software
      • Pioneers in Grid which has evolved into Cloud
      • Service governor leader* – “the brain of the cloud”
    • World-Class Customers
      • Hundreds of technology implementations
      • Dozens of Fortune 500 customers
    • Award-Winning Technology
      • Numerous industry awards and accolades
      • 2009 Top 100 VC Backed Companies – Red Herring
      • Patented technology
    • Global Reach
      • HQ in Chicago with offices across North America
      • Services delivered worldwide
      • Resellers in place worldwide
    * Gartner Q408
  • Univa Software Solutions
    • UniCloud
      • A workload management solution for matching workloads with available systems
      • Works with physical machines or virtual machines
      • A cloud building block
    • UniPortal
      • Self-service cloud portal
      • A cloud building block
    • Reliance
      • Rule management framework for leveraging cloud building blocks
      • Infrastructure and Application Service Governor
      • Allows for machine speed decisions
      • Can apply multi-variable, extremely complex rule systems for managing workload through entire lifecycle of the the work.
      • "the brains of the cloud"- where your real attention will be
  • Solution Key Features
    • Infrastructure & Application Governance
      • Policy- driven resource management
      • Application o riented
    • Automation
      • Dynamic provision (re)builds the application stack or image on-the-fly
      • Kits support application configuration
      • Native package management simplifies software updates
      • Cluster configuration sets up the required system services
      • Network provisioning
      • Workload Automation – Sun Grid Engine
    • Mobility
      • Dynamic provisioning avoids lock-in
      • Can manage and move applications between environments
        • On-premise bare metal
        • Private cloud using Oracle VM and other hypervisors
        • Cloud: Amazon EC2, Rackspace (soon)
  • What is a UniCluster Kit?
    • A Kit is a ‘meta-rpm’ package for installing software into a cluster
    • Kits contain one or many components
    • Special Kit and component rpms define plugins for configuring the software, pre/post install scripts and dependencies
    • Components contain a ‘unit’ of software that can be installed onto nodes
    • Components encapsulate packages for multiple Operating Systems and Architecture
    • Kits abstraction designed to support other package managers – example: rpm, ips.
    • Kits provide flexibility – install only what is needed on the cluster
    • Kits deploy into existing Operating System repository
    Kit ‘A’ Component A-1 Component A-2 ... Component A-k RPM 1 RPM 2 RPM 3 RPM N ... Kit ‘B’ Kits can be added to the cluster several ways: via a yum repository, ISO or physical media
  • Hardware and Software Profiles
    • Nodes are arranged in groups with profiles.
    • Hardware Profiles define physical & virtual hardware.
    • Software profiles define the ‘stack’ on the machine.
    • Software profiles can be changed without reprovisioning action
    • Hardware profile changes trigger reprovision .
  • UniCloud: CloudCluster
  • UniCloud: CloudBursting
  • UniCloud: Private Cloud
  • Service Provider Cloud
  • Univa Cloud Use Cases 5. The Same Kit is used for all environments. Result: Same Software Stack in house virtual and physical. As Public cloud software stack, Customer is not tied to one Public Cloud Provider 1. Kits are stored in the UniCloud Repo. One Kit for virtual, physical and public cloud. 2. Kits are ‘applications’ and can be installed in a local Virtual machine. 3. A Local Physical Machine 4. Or even a Public cloud such as EC2.
  • UniCloud 2.0 Policy Driven Live Migration SGE dispatches large job to new machine 3 hypervisor machines. SGE, Tortuga Framework configured in the Cluster Each physical machine is running two virtual machines SGE ‘sees’ 6 machines in the cluster. 10 small jobs arrive in the low priority ‘Q’ SGE dispatches small jobs to virtual machines Small jobs ‘fill’ all of the virtual machines. A large job arrives in the High ‘Q’ The Policy Engine collects data from SGE Qmaster The Policy Engine triggers on a rule noticing pending jobs in High Q. Policy Engine Directs Tortuga WS to ‘transfer node’ which moves VM to other hypervisor Policy Engine Does this twice. Policy Engine directs Tortuga WS to =‘re-provision to Physical node’ Tortuga automatically reprovisions and configures SGE. Policy Engine Tortuga DB Tortuga WS SGE Qmaster Policy WS High Q Low Q Oracle Cluster Filesystem ‘Heartbeat’ eexecd eexecd eexecd eexecd eexecd eexecd eexecd eexecd R1 R2 eexecd
  • Univa Value Add to HPC Customers
    • Increased Utilization
    • Share machines that were previously “reserved”
    • Run more workload in the same time
    • Accommodate priorities on-the-fly
    • Increased Optimization
    • Licenses can be fully utilized
    • Reduced “waste”
    • Lower Total Cost of Ownership
    • Reduced administration time
    • Reduced dedicated hardware
    • Reduced downtime
    Business Impact Rapid ROI via Cost Recovery
  • Extreme Business Applications
    • Integrated Business Platform
    • Oracle e-Business Suite
    • Features
      • Sun integrated networking, computing and storage infrastructure
      • Integrated batch workload management with Sun Grid Engine
      • Oracle RAC, Oracle e-Business Suite, Oracle Enterprise Linux & Oracle VM
      • Policy-driven, dynamic scaling and provisioning by Univa
  • Infrastructure Abstraction “Some Thoughts”
    • Computation Resources
    • Many virtualization technologies & products, such as VMware, Xen (XenServer, Oracle VM, etc.), Virtualbox, etc.
    • Management over massive VMs.
    • Storage/Communication
    • High throughput & high performance, but low cost?
    • I/O QoS
    • Isolation for different domains: Cisco Nexus v1000, Open vSwitch, etc.
    • I/O Virtualization in native implementation
    • Security??
  • Q&A Q&A
  • Sun Microsystems, a subsidiary of Oracle Corp. THANK YOU!