Cloud computing final
Upcoming SlideShare
Loading in...5
×
 

Cloud computing final

on

  • 1,821 views

complete idea about cloud computing

complete idea about cloud computing

Statistics

Views

Total Views
1,821
Slideshare-icon Views on SlideShare
1,819
Embed Views
2

Actions

Likes
1
Downloads
229
Comments
0

1 Embed 2

http://192.168.6.179 2

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Cloud Computing is a general term used to describe a new class of network based computing that takes place over the Internet, basically a step on from Utility Computing.In other words, this is a collection/group of integrated and networked hardware, software and Internet infrastructure (called a platform).Using the Internet for communication and transport provides hardware, software and networking services to clients.These platforms hide the complexity and details of the underlying infrastructure from users and applications by providing very simple graphical interface or API (Applications Programming Interface).
  • Examples:-Gmail, GoogleCalenderPayroll, HR, CRM etcSugarm CRM, IBM Lotus Live
  • Type 1: Ad-Hoc/CustomEach customer (or tenant) has there own instance of the application which can be customized on an individual basisLevel 1 SaaS is equivalent to application hostingType 2: ConfigurableA single application base is customized for each customer/tenantCustomization is deployed within each instance of the applicationDeployment of upgrades across the instance will require roll-out to each instanceType 3: Configurable, Multi-Tenant-EfficientA single application base and instance is customised for each customer/tenantCustomisation is deployed at run-time within each instance of the applicationSingle instance is more resource efficient than multiple instancesDeployment of upgrades made to a single instanceType 4: Scalable, Configurable, Multi-Tenant-EfficientUses a tenant load balancer to balance load between multiple instancesSimilar to a hypervisorShould provide superior scalability and efficiencyRequires deployment of upgrades to made to multiple instances
  • Platform-as-a-service in the cloud is defined as a set of software and product development tools hosted on the provider’s infrastructure. Developers create applications on the provider’s platform over the Internet. PaaS providers may use APIs, website portals or gateway software installed on the customer’s computer Force.com, (an outgrowth of Salesforce.com) and GoogleApps are examples ofPaaS. Developers need to know that currently, there are not standards for interoperability or data portability in the cloud. Some providers will not allow software created by their customers to be moved off the provider’s platform.
  • Benefits to the clients: 1. Stop worrying about heavy traffic and bandwidth requirements. 2. Pay as you go. 3. No need to buy high configuration servers from day one. 4. Low maintenance.
  • CRM=costumer relationship management
  • 1- A service-oriented architecture is essentially a collection of services. These services communicate with each other. The communication can involve either simple data passing or it could involve two or more services coordinating some activity. Some means of connecting services to each other is needed.2- An enterprise architecture (EA) is a conceptual blueprint that defines the structure and operation of an organization. The intent of an enterprise architecture is to determine how an organization can most effectively achieve its current and future objectives.3- middle ware : In the computer industry, middleware is a general term for any programming that serves to "glue together" or mediate between two separate and often already existing programs. A common application of middleware is to allow programs written for access to a particular database to access other databases4- The term software architecture intuitively denotes the high level structures of a software system. It can be defined as the set of structures needed to reason about the software system, which comprise the software elements, the relations between them, and the properties of both elements and relations.[1]The term software architecture also denotes the set of practices used to select, define or design a software architecture.
  • At the core, migration of an application into the cloud can happen in one ofseveral ways: Either the application is clean and independent, so it runs as is; orperhaps some degree of code needs to be modified and adapted; or the design(and therefore the code) needs to be first migrated into the cloud computingservice environment; or finally perhaps the migration results in the corearchitecture being migrated for a cloud computing service setting, this resultingin a new architecture being developed, along with the accompanying design andcode implementation. Or perhaps while the application is migrated as is, it is theusage of the application that needs to be migrated and therefore adapted andmodified.where P is the application before migration running in captive data center, P0C isthe application part after migration either into a (hybrid) cloud, P0l is the partof application being run in the captive local data center, and P0OFC is theapplication part optimized for cloud. If an enterprise application cannot bemigrated fully, it could result in some parts being run on the captive local datacenter while the rest are being migrated into the cloud—essentially a case of ahybrid cloud usage. However, when the entire application is migrated onto thecloud, then P0l is null. Indeed, the migration of the enterprise application P canhappen at the five levels of application, code, design, architecture, and usage. Itcan be that the P0C migration happens at any of the five levels without any P0lComponent.Technology reasons Economic and business reasons
  • The first stepof the iterative process of the seven-step model of migration is basically at theassessment level. Proof of concepts or prototypes for various approaches tothe migration along with the leveraging of pricing parameters enables oneto make appropriate assessments.These assessments are about the cost of migration as well as about the ROIthat can be achieved in the case of production version. The next process step is inisolating all systemic and environmental dependencies of the enterprise application components within the captive data center. This, in turn, yieldsa picture of the level of complexity of the migration. After isolation is complete,one then goes about generating the mapping constructs between what shallpossibly remain in the local captive data center and what goes onto the cloud.Perhaps a substantial part of the enterprise application needs to be rearchitected,redesigned, and reimplemented on the cloud. This gets in just aboutthe functionality of the original enterprise application. Due to this migration, itis possible perhaps that some functionality is lost. In the next process step weleverage the intrinsic features of the cloud computing service to augment ourenterprise application in its own small ways. Having done the augmentation, wevalidate and test the new form of the enterprise application with an extensivetest suite that comprises testing the components of the enterprise application onthe cloud as well. These test results could be positive or mixed. In the latter case,we iterate and optimize as appropriate. After several such optimizing iterations,the migration is deemed successful. Our best practices indicate that it is best toiterate through this Seven-Step Model process for optimizing and ensuring thatthe migration into the cloud is both robust and comprehensive.
  • Virtualization is a broad term (virtual memory, storage, network, etc)Focus for this course: platform virtualizationVirtualization basically allows one computer to do the job of multiple computers, by sharing the resources of a single hardware across multiple environmentsLife cycleResource SelectionResource Preparation VM Creation VM MigrationVM Termination PAGE162
  • Everything is virtualizedFull hardware emulationEmulation = latency
  • Within a Public Cloud (figure 3.1). Two different applications are hosted ina cloud. The role of the cloud integration middleware (say cloud-based ESB orinternet service bus (ISB)) is to seamlessly enable these applications to talk toeach other. The possible sub-scenarios include these applications can be ownedby two different companies. They may live in a single physical server but run ondifferent virtual machines.Homogeneous Clouds (figure 3.2). The applications to be integrated areposited in two geographically separated cloud infrastructures. The integrationmiddleware can be in cloud 1 or 2 or in a separate cloud.There is a need for data and protocol transformation and they get done bythe ISB. The approach is more or less compatible to enterprise applicationintegration procedure.Heterogeneous Clouds (figure 3.3). One application is in public cloud andthe other application is private cloud.As described above, this is the currently dominating scene for cloudintegration. That is, businesses are subscribing to popular on-demand enterprisepackages from established providers such as Salesforce.com and RamcoSystems (http://www.ramco.com/)’s customer relationship management(CRM), NetSuite’s (http://www.netsuite.com) enterprise resource planning(ERP), etc. The first two scenarios will become prevalent once there are severalcommercial clouds and cloud services become pervasive. Then service integrationand composition domains will become an important and incredible factorfor global computing.Cloud 1 ISB Cloud 2FIGURE 3.2. Across Homogeneous Clouds.Public CloudISBPrivate CloudFIGURE 3.3. Across Heterogeneous Clouds
  • Service-level agreement provides a framework within which both seller and buyer of a service can pursue a profitable service business relationship. SLA can be modeled using web service-level agreement (WSLA) language specification.
  • Google file system Internet introduced a new challenge in the form web logs, web crawler’s data: large scale “peta scale”But observe that this type of data has an uniquely different characteristic than your transactional or the “customer order” data : “write once read many (WORM)” ; Privacy protected healthcare and patient information; Historical financial data; Other historical data Google exploited this characteristics in its Google file system (GFS)
  • There is large overhead in starting a Map-Reduce job, but once begun, the job can be completed rapidly (relative to conventional approaches).Map-Reduce requires writing two functions: a mapper and a reducer. These functions accept data as input and then return transformed data as output. The functions are called repeatedly, with subsets of the data, with the output of the mapper being aggregated and then sent to the reducer.
  • Determine if the problem is parallelizable and solvable using MapReduce (ex: Is the data WORM?, large data set).Design and implement solution as Mapper classes and Reducer class. Compile the source code with hadoop core.Package the code as jar executable.Configure the application (job) as to the number of mappers and reducers (tasks), input and output streamsLoad the data (or use it on previously available data)Launch the job and monitor.Study the result.Detailed steps.
  • Application: a set of virtual machines and data volumes connected by a virtual network to be deployed at IaaS levelFederation: common entry point for multiple cloud providers

Cloud computing final Cloud computing final Presentation Transcript

  • Cloud ComputingGets its name as a metaphor for the internet Omer K. Jasim Ph.D. qualifier exam FCIS-ASU’2013
  • Agenda FCSI, ASU, Egypt. Omer K. Jasim 1. Cloud Computing Definitions 2. Cloud Computing Layers 2.1 Cloud Architecture 2.2 Cloud Data Center 3. Main Cloud Process 3.1 Cloud Migration 3.2 Virtualization 4. Integration of Cloud Application 5. SLA’s in Cloud Computing 6. Workflow Management System 7. Cloud Computing Parallelism 7.1 Map-Reduce Model 7.2 Cloud Federation 8. Dissertation with this topics 9. References
  • Definitions • A large pool of easily usable and accessible virtualized resources (such as hardware, development platforms and/or services). “ Stallman 2011”. • A parallel and distributed computing system consisting of a collection of inter-connected and virtualized computers that are dynamically provisioned and presented as one or more unified computing resources based on service-level agreements (SLA) established through negotiation between the service provider and consumers. “Nicolas 2013 FCSI, ASU, Egypt. Omer K. Jasim
  • Cloud computing:- is the provision of dynamically scalable and often virtualized resources as a service over the internet (public cloud) or intranet (private cloud). The “Cloud” is the default symbol of the internet in diagrams. Cloud Computing The broader term of “Computing” encompasses: - no. of Computations - Coordination logic - Storage Cloud Computing is about moving computing from the single desktop pc/data centers to the internet. FCSI, ASU, Egypt. Omer K. Jasim Definition (cont.) set up an account with the provider, start and configure a virtual cluster with one or more nodes for computation, and return the results to your local machine
  • Agenda FCSI, ASU, Egypt. Omer K. Jasim 1. Cloud Computing Definitions 2. Cloud Computing Layers 2.1 Cloud Architecture 2.2 Cloud Data Center 3. Main Cloud Process 3.1 Cloud Migration 3.2 Virtualization 4. Integration of Cloud Application 5. SLA’s in Cloud Computing 6. Workflow Management System 7. Cloud Computing Parallelism 7.1 Map-Reduce Model 7.2 Cloud Federation 8. Dissertation with this topics 9. References
  • Cloud Computing Layers Computer, phones, Brow ser. Cloud Services Networks, Virtualization FCSI, ASU, Egypt. Omer K. Jasim Client Applications Platform Infrastructure Server
  • 1- Clients  Hardware and/or software  Essentially useless without the cloud Ex: Laptop & phones, Thin and Thick computer. FCSI, ASU, Egypt. Omer K. Jasim
  • Cloud layers (cont.) FCSI, ASU, Egypt. Omer K. Jasim Application Platform Infrastructure Cloud Services
  • 2- Cloud Service Data Centers Clusters Storage Other Grids/Clouds Virtualization VM Management & Deployment Amazon S3, EC2 OpenNebula, Eucalyptus Mosso Web 2.0 Interface Programming API Scripting & Programming Languages Google AppEngine Microsoft Azure Manjrasoft Aneka Google Apps (Gmail, Docs,…) Salesforce.com Infrastructure as a Service(IaaS) Platform as a Service(PaaS) Software as a Service(SaaS) QoSMetering SLA NegotiationAdmissionControl PricingandBilling Accounting FCSI, ASU, Egypt. Omer K. Jasim SPI
  • 2.1.Software as a Services (SaaS) - SaaS is a model of software deployment where an application is hosted as a service provided to customers cross the Internet. -Features: • Multi-tenancy, Multi -instance – One application instance may be serving hundreds of companies – Opposite of multi-instance where each customer is provisioned their own server running one instance • Metadata driven configurability – Instead of customizing the application for a customer (requiring code changes), one allows the user to configure the application through metadata FCSI, ASU, Egypt. Omer K. Jasim
  • SaaS Maturity Levels • Level 1: Ad-Hoc/Custom • Level 2: Configurable • Level 3: Configurable, Multi- Tenant-Efficient • Level 4: Scalable, Configurable, Multi -Tenant-Efficient Source: Microsoft MSDN Architecture Center FCSI, ASU, Egypt. Omer K. Jasim • Level 1: Ad-Hoc/Custom • Level 2: Configurable • Level 3: Configurable, Multi- Tenant-Efficient • Level 4: Scalable, Configurable, Multi -Tenant-Efficient
  • 2.2. Platform as a Services (PaaS) – Providing all the facilities necessary to support the complete process of building and delivering web applications and services, all available over the Internet. – Entirely virtualized platform that includes one or more servers, operating systems and specific applications. Examples:- Middleware, VM, etc.. FCSI, ASU, Egypt. Omer K. Jasim
  • 2.3.Infrastrcture as a Services (IaaS) – Provision model in which an organization outsources the equipment used to support operations, including storage, hardware, servers and networking components. – Also known as Hardware as a Service (HaaS). – Service provider owns the equipment; responsible for housing, running and maintaining it. – Client typically pays on a per-use basis. Examples:- IBM Blue house, VMWare, Amazon EC2, Microsoft Azure Platform, Sun Parascale and more FCSI, ASU, Egypt. Omer K. Jasim
  • (Customer vs. Provider) view XaaS FCSI, ASU, Egypt. Omer K. Jasim
  • 3- Server  Hardware/Software  Specialized for cloud  OS, multi core processors  Network of virtualized computers FCSI, ASU, Egypt. Omer K. Jasim
  • Cloud Architecture The Cloud Computing Architecture of a cloud solution is the structure of the system, which comprise enterprise arch. and cloud resources, services, middleware, and software components, SOA, the externally visible properties of those, and the relationships between them FCSI, ASU, Egypt. Omer K. Jasim Collection of services , communicate with each other or two or more services coordinate to preform some activities Any program supported “glue together”, allow programs written for access to a particular DB. Structure and operation of an organization Set of structures needed to reason about the system software
  • Deployment Models • Public cloud (off-site and remote) : resources dynamically provisions on the internet • A private cloud : consolidating shared services on virtualized hardware deployed from a primary datacenter to serve local and remote users (intranet). • A hybrid cloud environment consists of some portion of computing resources on-site (on premise) and off-site (public cloud). • A community cloud is formed when several organizations with similar requirements share common infrastructure. FCSI, ASU, Egypt. Omer K. Jasim
  • Agenda FCSI, ASU, Egypt. Omer K. Jasim 1. Cloud Computing Definitions 2. Cloud Computing Layers 2.1 Cloud Architecture 2.2 Cloud Data Center 3. Main Cloud Process 3.1 Cloud Migration 3.2 Virtualization 4. Integration of Cloud Application 5. SLA’s in Cloud Computing 6. Workflow Management System 7. Cloud Computing Parallelism 7.1 Map-Reduce Model 7.2 Cloud Federation 8. Dissertation with this topics 9. References
  • Cloud Data Center FCSI, ASU, Egypt. Omer K. Jasim • Data center: large server farms + data warehouses. - managed infrastructure - Supported transmission from web hosting to cloud computing - individual web/content providers: must provision for peak load. • web hosting: third party provides and owns the (server farm) infrastructure, hosting web services for content providers • “server consolidation” via virtualization and “ Collaboration mechanism “
  • FCSI, ASU, Egypt. Omer K. Jasim WAN Network Functional Servers WAN Edge Data Center Core/ Gateway Customer Edge L2 Aggregator Server Rack Virtual Server Application VM/Server Control Server Hypervisor Virtual Subnet/VLAN 1 Virtual Subnet/VLAN 2 Application Virtual Machine (VM) Application Virtual Machine (VM) Application Virtual Machine (VM) Application Virtual Machine (VM) Application Virtual Machine (VM) Application Virtual Machine (VM) Application Virtual Machine (VM) Application Virtual Machine (VM) Provisioning, Billing, Security, Load Balancing, Monitoring, Auditing, Logging, and ETC. Functional Servers Collaboration level Basic components
  • Agenda FCSI, ASU, Egypt. Omer K. Jasim 1. Cloud Computing Definitions 2. Cloud Computing Layers 2.1 Cloud Architecture 2.2 Cloud Data Center 3. Main Cloud Process 3.1 Cloud Migration 3.2 Virtualization 4. Integration of Cloud Application 5. SLA’s in Cloud Computing 6. Workflow Management System 7. Cloud Computing Parallelism 7.1 Map-Reduce Model 7.2 Cloud Federation 8. Dissertation with this topics. 9. References
  • Main Process - Migration Process - Transmitted the enterprise application from the desktop to open cloud environment. - Virtualization - Abstraction of computer resources. - Hides the physical characteristics of computing resources from their users, be they applications, or end users. FCSI, ASU, Egypt. Omer K. Jasim
  • Migration to the Cloud - Move the enterprise application to the open cloud environment, captive cloud data center - Implemented in one of the several ways: (code, architecture, and usage ) - The migration of an enterprise application is based capture by the following: P P’C+P’l P’OFC+P’l FCSI, ASU, Egypt. Omer K. Jasim
  • FCSI, ASU, Egypt. Omer K. Jasim Optimiz e Assess Test Augment Re- architect Map Isolate Start End Migration Life Cycle
  • Agenda FCSI, ASU, Egypt. Omer K. Jasim 1. Cloud Computing Definitions 2. Cloud Computing Layers 2.1 Cloud Architecture 2.2 Cloud Data Center 3. Main Cloud Process 3.1 Cloud Migration 3.2 Virtualization 4. Integration of Cloud Application 5. SLA’s in Cloud Computing 6. Workflow Management System 7. Cloud Computing Parallelism 7.1 Map-Reduce Model 7.2 Cloud Federation 8. Dissertation with this topics 9. References
  • Virtualization • Virtualization:- the abstraction of computer resources. • Virtualization hides the physical characteristics of computing resources from their users, be they applications, or end users. • This includes making a single physical resource (such as a server, an operating system, an application, or storage device) appear to function as multiple virtual resources; it can also include making multiple physical resources (such as storage devices or servers) appear as a single virtual resource. FCSI, ASU, Egypt. Omer K. Jasim
  • VM-Architecture Hardware Virtual Machine Monitor (VMM) / Hypervisor Guest OS (Linux) Guest OS (NetBSD) Guest OS (Windows) VM VM VM AppApp AppAppApp Xen VM-Ware UML Denali etc. Hypervisor is a software program that manages multiple operating systems (or multiple instances of the same operating system) on a single computer system. The hypervisor manages the system's processor, memory, and other resources to allocate what each operating system requires. Hypervisors are designed for a particular processor architecture and may also be called virtualization managers. FCSI, ASU, Egypt. Omer K. Jasim
  • Full Virtualization • Provides a complete simulation of the underlying hardware. • The result is a system in which all software (including all OS’s) capable of execution on the raw hardware can be run in the virtual machine. • Comprehensively simulate all computing elements as instruction set, main memory, interrupts, exceptions, and device access. • Full virtualization is only possible given the right combination of hardware and software elements. FCSI, ASU, Egypt. Omer K. Jasim
  • Para virtualization Do not try to emulate everything , • Work as a guard • Pass safe instructions directly to CPU and device • Guests have some exposure to the hardware • Better performance • Need to slightly modify guest OS, but no need to modify applications • Xen, Sun Logical Domains FCSI, ASU, Egypt. Omer K. Jasim
  • Cloud based virtualization • PROVIDER - Support cloud to improve IT efficiency, agility and reliability. - Delivers everything IT needs to build, operate, rent, and manage their cloud. . CUSTOMER - Helps customers evolve technical foundations, organizational models, operational processes and financial measures to establish both a cloud infrastructure and cloud operations model. FCSI, ASU, Egypt. Omer K. Jasim
  • Agenda FCSI, ASU, Egypt. Omer K. Jasim 1. Cloud Computing Definitions 2. Cloud Computing Layers 2.1 Cloud Architecture 2.2 Cloud Data Center 3. Main Cloud Process 3.1 Cloud Migration 3.2 Virtualization 4. Integration of Cloud Application 5. SLA’s in Cloud Computing 6. Workflow Management System 7. Cloud Computing Parallelism 7.1 Map-Reduce Model 7.2 Cloud Federation 8. Dissertation with this topics 9. References
  • Integration of Cloud Computing • Important factor in the cloud. • Data enterprise and applications must be linked up via one or more standard integration platform . • Integration middle (I-MW or C-MW) ware simplify the mapping between Application to Application (A2A), Application to Cloud (A2C), Cloud to Cloud(C2C). FCSI, ASU, Egypt. Omer K. Jasim
  • Cloud -MW • Cloud middleware will be made available as a service. • Due to varying integration requirements and scenarios, there are a number of middleware technologies and products. • For performance sake, clusters, fabrics, grids, and federations of hubs, brokers, and buses are being leveraged. FCSI, ASU, Egypt. Omer K. Jasim
  • Cloud-MW FCSI, ASU, Egypt. Omer K. Jasim App1 App2 ISB Pub. CloudISB Cloud CloudISB Application App2App App2C C2C
  • FCSI, ASU, Egypt. Omer K. Jasim Understandi ng Testing Design Definition Integration life cycle Understanding the source system or the application on this system, target system Acquire information from previous phase to define the metadata, physical attribute, relation ship between objects Determined the data extract from the one or more system to target system Assuring the integration is done
  • Agenda FCSI, ASU, Egypt. Omer K. Jasim 1. Cloud Computing Definitions 2. Cloud Computing Layers 2.1 Cloud Architecture 2.2 Cloud Data Center 3. Main Cloud Process 3.1 Cloud Migration 3.2 Virtualization 4. Integration of Cloud Application 5. SLA’s in Cloud Computing 6. Workflow Management System 7. Cloud Computing Parallelism 7.1 Map-Reduce Model 7.2 Cloud Federation 8. Dissertation with this topics 9. References
  • Service Level Agreement in Cloud • Framework for provider, costumer . • Negotiation between service consumer and service provider, records common understanding about services. • Guarantee multiple QoS parameters across layers, highly dynamic environments, resources heterogeneity, Xaas service definition. • Capacity Planning: - dynamic workload amount, customer satisfaction level, • Maximize the profit: by reducing the resource cost (the number and type of initiated virtual machines and penalties cost). FCSI, ASU, Egypt. Omer K. Jasim
  • SLA-life Cycle FCSI, ASU, Egypt. Omer K. Jasim
  • FCSI, ASU, Egypt. Omer K. Jasim • Infrastructure SLA. The infrastructure provider manages and offers guarantees on availability of the infrastructure, namely, server machine, power, network connectivity, and so on. • Application SLA. In the application co-location hosting model, the server capacity is available to the applications based solely on their resource demands. SLA-Type
  • SLA Hierarchy SLA at an upper layer depends on SLAs at lower layers. FCSI, ASU, Egypt. Omer K. Jasim
  • Agenda FCSI, ASU, Egypt. Omer K. Jasim 1. Cloud Computing Definitions 2. Cloud Computing Layers 2.1 Cloud Architecture 2.2 Cloud Data Center 3. Main Cloud Process 3.1 Cloud Migration 3.2 Virtualization 4. Integration of Cloud Application 5. SLA’s in Cloud Computing 6. Workflow Management System 7. Cloud Computing Parallelism 7.1 Map-Reduce Model 7.2 Cloud Federation 8. Dissertation with this topics 9. References
  • FCSI, ASU, Egypt. Omer K. Jasim Cloud Workflow (CW) • A process consist of a series of steps that simplifies the complexity of execution and management of application. • Data Management • Data replication • Data Placement • Intermediate data storage
  • FCSI, ASU, Egypt. Omer K. Jasim
  • FCSI, ASU, Egypt. Omer K. Jasim CW: Data Management • New features in cloud computing – Independent of users and provider – Cost driven • computation cost, storage cost, data transfer cost – Data dependency • Task – data, data – data. • Some research issues – Data partition, placement, replication, synchronisation. Example : Aneka, open Nebula .
  • FCSI, ASU, Egypt. Omer K. Jasim CW: Data Placement • Data Placement: to decide where to store the application data in the distributed data centres • Aims: – Reduce data movement – Reduce task waiting time • Strategy: – Data dependency: dataset – dataset
  • FCSI, ASU, Egypt. Omer K. Jasim CW: Data Replication • Data replication: for one dataset, store several copies in different places (data centres) • Aims: – Increase data security – Fast data access – Reduce data movement • Strategy: – Dynamic replication.
  • FCSI, ASU, Egypt. Omer K. Jasim CW: Intermediate Data Storage • Intermediate data storage is especially importance in scientific workflows • Aim: – Reduce system cost • Strategy: – Intermediate data can be regenerated with data provenance information – Selectively store some key intermediate datasets
  • Agenda FCSI, ASU, Egypt. Omer K. Jasim 1. Cloud Computing Definitions 2. Cloud Computing Layers 2.1 Cloud Architecture 2.2 Cloud Data Center 3. Main Cloud Process 3.1 Cloud Migration 3.2 Virtualization 4. Integration of Cloud Application 5. SLA’s in Cloud Computing 6. Cloud Workflow system 7. Cloud Computing Parallelism 7.1 Map-Reduce Model 7.2 Cloud Federation 8. Dissertation with this topics 9.References
  • Cloud Computing Parallelism • “Cloud” refers to large Internet services like Google, Yahoo, etc. that run on 10,000’s of machines • - Want to parallelize across hundreds/thousands of CPUs – How to parallelize – How to distribute -Map-Reduce model based to answer the above questions. FCSI, ASU, Egypt. Omer K. Jasim
  • Map –Reduce • Map-Reduce is a data processing approach(data parallel programming model) that presents a simple programming model for processing highly parallelizable data sets. It is implemented as a cluster, with many nodes working in parallel on different parts of the data. • Pioneered by Google – Processes 20 petabytes of data per day • Popularized by open-source Hadoop project – Used at Yahoo!, Facebook, Amazon, … FCSI, ASU, Egypt. Omer K. Jasim
  • Data Distribution • Input files are split into M pieces on distributed file system – Typically ~ 64 MB blocks • Intermediate files created from map tasks are written to local disk • Output files are written to distributed file system
  • Assigning Tasks • Many copies of user program are started • Tries to utilize data localization by running map tasks on machines with data • One instance becomes the Master • Master finds idle machines and assigns them tasks
  • FCSI, ASU, Egypt. Omer K. Jasim Master worker1 Worker nworker3worker2 Data tasks Task1 Task2 . . . Task n Data distributed Assigning to the tasks
  • Example for MapReduce • Page 1: computer study is good • Page 2: cloud computing is good • Page 3: my study in cloud computing • ……. • ….. • Page n.: ….. Any data ?
  • Map output • Worker 1: – (computer 1), (study 1), (is 1), (good 1). • Worker 2: – (cloud 1), (computing 1), (is 1), (good 1). • Worker 3: – (My 1), (study 1), (in 1), (cloud 1), (computing 1).
  • Map Operation MAP: Input data  <key, value> pair Data Collection: split1 web 1 weed 1 green 1 sun 1 moon 1 land 1 part 1 web 1 green 1 … 1 KEY VALUE Split the data to Supply multiple processors Data Collection: split 2 Data Collection: split n Map …… Map web 1 weed 1 green 1 sun 1 moon 1 land 1 part 1 web 1 green 1 … 1 KEY VALUE web 1 weed 1 green 1 sun 1 moon 1 land 1 part 1 web 1 green 1 … 1 KEY VALUE web 1 weed 1 green 1 sun 1 moon 1 land 1 part 1 web 1 green 1 … 1 KEY VALUE web 1 weed 1 green 1 sun 1 moon 1 land 1 part 1 web 1 green 1 … 1 KEY VALUE … FCSI, ASU, Egypt. Omer K. Jasim
  • Reduce Input • Worker 1: – (computer 1) • Worker 2: – (is 1), (is 1) • Worker 3: – (good 1), (good 1) • Worker 4: – (study 1), (study1) • Worker 5: – (cloud 1), (cloud 1) • Worker 6: – (computing 1), (computing 1) •Worker 6: - (My 1)
  • Reduce Output • Worker 1: – (computer 1) • Worker 2: – (is 2) • Worker 3: – ( good 2) • Worker 4: – (study 2) • Worker 5: – (cloud 2)
  • Reduce Operation MAP: Input data  <key, value> pair REDUCE: <key, value> pair  <result> Reduce Reduce Reduce Data Collection: split1 Split the data to Supply multiple processors Data Collection: split 2 Data Collection: split n Map Map …… Map … FCSI, ASU, Egypt. Omer K. Jasim
  • CountCountCount worker1 Worker n Parse-hash worker2 Map <key, 1> Reducers (say, Count) P-0000 P-0001 P-0002 , count1 , count2 ,count3 FCSI, ASU, Egypt. Omer K. Jasim Work 3
  • Parallel Execution No. of CPU. Mapping
  • Fault Tolerance • Network Failure: – Detect failure via periodic heartbeats – Re-execute completed and in-progress map tasks – Re-execute in progress reduce tasks – Task completion committed through master • Master failure: – Could handle, but don't yet (master failure unlikely)
  • Agenda FCSI, ASU, Egypt. Omer K. Jasim 1. Cloud Computing Definitions 2. Cloud Computing Layers 2.1 Cloud Architecture 2.2 Cloud Data Center 3. Main Cloud Process 3.1 Cloud Migration 3.2 Virtualization 4. Integration of Cloud Application 5. SLA’s in Cloud Computing 6. Cloud Workflow system 7. Cloud Computing Parallelism 7.1 Map-Reduce Model 7.2 Cloud Federation 8. Dissertation with this topics 9.References
  • Cloud Federation • Cloud Federation is a collection of individual Cloud providers, which collaborate by trading resources (e.g. computing, storage). • Desired feature of Cloud – Illusion of infinite computing resources • Resources available in a single data center are limited – A large demand may put pressure in the data center capacity – One possible source for additional resources is idling resources from other providers FCSI, ASU, Egypt. Omer K. Jasim
  • - A user submit an application to the federation, the federation select provider to run application. FCSI, ASU, Egypt. Omer K. Jasim Cloud federation provider1 provider2 Provider 3 Provider n
  • Cloud Federation Motivations • Obtain extra resources from other members (Outsourcing) – Avoid losing customers – Avoid losing reputation by violating SLAs • Lease idle resources (Contributing to Federation – Avoid wasting their non-storable compute resources • Supplying resources in specific geographic locations – Low-latency access regardless of location – meet regulations in place for the customers • Disaster recovery FCSI, ASU, Egypt. Omer K. Jasim
  • Cloud storage • Cloud storage enables to “throw” data into the cloud and without worrying about how it is stored or backing it up. When you need it again, you simply reach into the cloud and grab it. • User don’t know how it is stored, where it is stored, or what has happened to all the pieces of hardware between the time you put it in the cloud and the time you retrieved it. FCSI, ASU, Egypt. Omer K. Jasim
  • Yesterday’s storage Local storage Storage Servers FCSI, ASU, Egypt. Omer K. Jasim
  • Today’s storage e.g. Google Laptop Cloud Storage FCSI, ASU, Egypt. Omer K. Jasim
  • Internet-service FS: GoogleFS, HadoopFS, CloudStore, ... Custom Storage: Facebook Haystack Photo Store, Microsoft StarTrack (Map Apps), Amazon S3, EBS, ... Key-Value Store: Cassandra, Voldemort, ... Structured Storage: Yahoo! PNUTS, Google BigTable, Hbase, ... + Replication + Scale-up + Migration + ... FCSI, ASU, Egypt. Omer K. Jasim
  • Agenda FCSI, ASU, Egypt. Omer K. Jasim 1. Cloud Computing Definitions 2. Cloud Computing Layers 2.1 Cloud Architecture 2.2 Cloud Data Center 3. Main Cloud Process 3.1 Cloud Migration 3.2 Virtualization 4. Integration of Cloud Application 5. SLA’s in Cloud Computing 6. Cloud Workflow system 7. Cloud Computing Parallelism 7.1 Map-Reduce Model 7.2 Cloud Federation 8. Our Dissertation with this topics 9. References
  • Our Dissertation - Cloud computing environment introduce more features in on-demand scalability/elasticity, affordability, global-scale accessibility. - All above features are done in open environments, therefore it vulnerable to many types of attacks. - Our dissertation attempt to provide a hybrid cryptography model based on CipherCloud and Quantum Cryptography to present a new concept in cloud security world. FCSI, ASU, Egypt. Omer K. Jasim
  • References 1- George Reese, “Cloud Application Architectures” USA, 2009. 2- Zaigham Mahmood and Richard Hill, Cloud Computing for Enterprise Architectures. ©Springer-Verlag London Limited 2011 3- Rajkumar Buyya and James Broberg, CLOUD COMPUTING “Principles and Paradigms”, 2011 by John Wiley. 4- Anthony T. Velte and Robert Elsenpeter, Cloud Computing: “A Practical Approach”© 2010 by The McGraw-Hill Companies FCSI, ASU, Egypt. Omer K. Jasim