This document discusses the key aspects of cloud computing. It begins by outlining the massive scale of today's clouds, with companies like Facebook, Microsoft, and Amazon operating clouds with tens or hundreds of thousands of servers. It then describes the main categories of cloud services - Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS) - which allow on-demand access to computing resources. Finally, it notes that clouds are data-intensive in nature and have enabled new programming paradigms like MapReduce that make it easy to write highly parallel programs for large datasets.
Cloud computing has grown tremendously and is projected to continue rapid growth. It provides massive computing resources that can be accessed on-demand and paid for through utility-based pricing models. There are various types of cloud services including Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS). Cloud computing enables data-intensive applications through massive data storage and the ability to run large-scale distributed computing frameworks.
There are many cloud providers that offer infrastructure, platform, and software services. Clouds can be public, accessible to any paying customer, or private and only accessible internally. Customers save significant time and money using cloud services by gaining rapid access to flexible computing resources without large upfront investments.
Cloud computing refers to applications and services delivered over the Internet. It allows users to access files, personal data, and applications from any device with Internet access. Key benefits include reduced costs, flexibility, and scalability. Challenges include availability of data if a provider fails, data lock-in to specific providers, and performance unpredictability. However, opportunities exist through standardization, encryption, and improved virtualization support. The future of cloud computing is promising with many companies investing heavily and forecasts of widespread adoption over the next few years.
An Introduction to Data Intensive ComputingCollin Bennett
This document provides an introduction to data intensive computing. It discusses how data and computing are growing exponentially due to improvements in instruments and technologies. This is creating new paradigms of data intensive science and computing. The document then discusses how cloud computing models like utility clouds (e.g. Amazon) and data clouds are facilitating data intensive computing by providing scalable resources and platforms for storing, managing and processing large amounts of data. Key concepts covered include virtualization, infrastructure as a service (IaaS), and MapReduce as a programming model for distributed computing on big data.
Cloud computing is a type of Internet-based computing that provides shared computer processing resources and data to computers and other devices on demand. It is a model for enabling ubiquitous, on-demand access to a shared pool of configurable computing resources (e.g., computer networks, servers, storage, applications and services),
Cloud computing has grown tremendously and is projected to continue rapid growth. It provides massive computing resources that can be accessed on-demand and paid for through utility-based pricing models. There are various types of cloud services including Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS). Cloud computing enables data-intensive applications through massive data storage and the ability to run large-scale distributed computing frameworks.
There are many cloud providers that offer infrastructure, platform, and software services. Clouds can be public, accessible to any paying customer, or private and only accessible internally. Customers save significant time and money using cloud services by gaining rapid access to flexible computing resources without large upfront investments.
Cloud computing refers to applications and services delivered over the Internet. It allows users to access files, personal data, and applications from any device with Internet access. Key benefits include reduced costs, flexibility, and scalability. Challenges include availability of data if a provider fails, data lock-in to specific providers, and performance unpredictability. However, opportunities exist through standardization, encryption, and improved virtualization support. The future of cloud computing is promising with many companies investing heavily and forecasts of widespread adoption over the next few years.
An Introduction to Data Intensive ComputingCollin Bennett
This document provides an introduction to data intensive computing. It discusses how data and computing are growing exponentially due to improvements in instruments and technologies. This is creating new paradigms of data intensive science and computing. The document then discusses how cloud computing models like utility clouds (e.g. Amazon) and data clouds are facilitating data intensive computing by providing scalable resources and platforms for storing, managing and processing large amounts of data. Key concepts covered include virtualization, infrastructure as a service (IaaS), and MapReduce as a programming model for distributed computing on big data.
Cloud computing is a type of Internet-based computing that provides shared computer processing resources and data to computers and other devices on demand. It is a model for enabling ubiquitous, on-demand access to a shared pool of configurable computing resources (e.g., computer networks, servers, storage, applications and services),
This document provides an overview of cloud computing. It begins with Gartner's definition of cloud computing as delivering elastic IT capabilities as a service using internet technologies. The document then discusses the history of cloud computing from John McCarthy's vision in 1961 to modern cloud services from Amazon, Google, and Microsoft. It explores how cloud computing solves the scalability problem by allowing resources to rapidly scale up or down as needed. The document outlines the key cloud service models of IaaS, PaaS, and SaaS and provides examples. It also shares success stories of companies like Zynga and Netflix leveraging cloud computing. Finally, it discusses potential cost savings and benefits of cloud computing through case studies.
Introduction to Cloud Computing and Big Datawaheed751
This document provides an introduction to cloud computing and big data. It defines cloud computing as a model for providing scalable computing resources over the internet with minimal management. The key characteristics of cloud computing include on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service. Dynamic provisioning allows cloud resources to scale up and down based on demand. This helps solve the problems of underutilization and overload in traditional systems with static capacity. The document also discusses how dynamic provisioning can be used in multi-tier web applications running on cloud infrastructure.
Course 3 : Types of data and opportunities by Nikolaos DeligiannisBetacowork
This document discusses big data and opportunities related to different types of data. It covers challenges of big data including volume, velocity, variety and veracity. It also discusses value that can be extracted from data. The document outlines static versus real-time data and structured versus unstructured data. Examples of applying machine learning techniques like regression, classification, clustering and dimensionality reduction are provided. The introduction to cloud computing discusses public, private and hybrid clouds and features of cloud infrastructure.
The document discusses cloud computing, providing definitions and describing its evolution, service models (SaaS, PaaS, IaaS), deployment models (public, private, hybrid), and advantages/disadvantages. It defines cloud computing as on-demand access to shared pools of configurable computing resources via the internet. The three main service models are software as a service (SaaS), platform as a service (PaaS), and infrastructure as a service (IaaS). Public clouds are hosted by third parties, private clouds are operated solely by a single organization, and hybrid clouds combine public and private.
Google not all clouds are created equal - sap sapphire 2014 (1)David Torres
Google Cloud Platform is built using Google's globally connected infrastructure that has been optimized over 15+ years for scalability, performance, and quality. It provides data processing, storage, and analytics services like Compute Engine, BigQuery, and Cloud Storage. Customers can use these services to build and host applications, process vast amounts of data using MapReduce/Hadoop, and perform digital marketing analytics on large datasets.
This document summarizes the hype and growth around cloud computing, listing major cloud providers like AWS, Microsoft Azure, and Google Cloud. It notes that cloud computing is projected to grow from $40.7 billion in 2010 to $241 billion by 2020. The document also categorizes clouds as either public (accessible to any paying customer) or private (only accessible to company employees). It provides examples of how public clouds from AWS and Google charge for services like compute hours and storage. Finally, it highlights how several large companies have significantly reduced their IT costs and improved provisioning times by leveraging cloud computing.
Cloud computing allows consumers and businesses to use applications without installation and access files from any internet-connected device. It provides shared computing resources over a network on-demand as a utility. There are concerns around security and privacy as cloud providers have control over user data. However, new approaches around information-centric security aim to give more control to users. A shift to cloud computing could benefit large internet companies while traditional software producers may face challenges adapting. It remains to be seen if cloud computing will ultimately become the dominant IT model.
This document provides an overview of cloud computing from the perspective of UC Berkeley's Reliable Adaptive Distributed Systems (RAD) Lab. It discusses what is new about cloud computing, challenges and opportunities, considerations for moving workloads to the cloud, and the RAD Lab's experiences using public and private clouds. Some key points:
- Cloud computing enables on-demand access to computing resources without long-term commitments, allowing for flexible scaling up or down. This transfers risk from users to providers.
- Challenges include lock-in, availability, data transfer bottlenecks, and policy issues. Opportunities include risk transfer enabling new scenarios, standardization, and pay-as-you-go licensing models.
This document provides an overview of cloud computing concepts including:
- The key characteristics of cloud computing including on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service.
- The roots of cloud computing in technologies like virtualization, distributed computing, web services, and utility computing.
- The different service models of cloud computing including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).
This document discusses various topics related to cloud computing including big data and cloud, cloud as a utility, service availability, performance predictability, providing elasticity, data confidentiality, data lock-in, optimizing data placement and transfer, green cloud, emerging opportunities, and more research questions. It notes that while cloud computing provides benefits, there are also open challenges regarding reliability, security, flexibility and optimization that require further research.
Its a complete very concise yet effective presentation on cloud computing. This is an emerging technique in developing countries like african countries.
The document provides an overview of using Amazon Web Services (AWS) for high-performance computing (HPC) clusters. It discusses how AWS enables scientists to build HPC clusters on demand that can scale up and down based on workload needs. Specific solutions and services mentioned include Alces Flight for launching ready-to-compute HPC clusters on AWS in minutes, the AWS Spot Market for accessing spare computing capacity at low costs, and examples of using AWS for scientific workloads like satellite image analysis and computational fluid design simulations.
This document provides an introduction and overview of cloud computing. It defines cloud computing as applications and services delivered over the internet, as well as the hardware and systems that provide those services. The document discusses common cloud services like SaaS, utility computing using Amazon EC2 as an example, and the key characteristics of cloud computing. It also covers challenges of cloud computing like availability, data security, and performance unpredictability, as well as the growth and future potential of cloud computing.
cloud computing - concepts and technologies and mechanisms of tackling problems in cloud
you plz ignore who created it , plz focus on problem oriented points
This document discusses cloud computing concepts, technologies, and business implications. It provides an introduction to cloud models like IaaS, PaaS, and SaaS and demonstrates cloud capabilities through examples of Amazon AWS, Google App Engine, and Windows Azure. The document also discusses enabling technologies for cloud computing like virtualization and programming models for big data like MapReduce and Hadoop.
Cloud computing provides on-demand access to shared computing resources like networks, servers, storage, applications and services over the internet. It has seen rapid growth in recent years. There are different service models like Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS) depending on what capabilities are provided to the user. Cloud computing can be deployed using private, public, hybrid or community models depending on who manages the infrastructure and who has access to it. While cloud computing provides benefits like flexibility, scalability and cost savings, concerns around security, privacy and reliability remain challenges to adoption.
An improved modulation technique suitable for a three level flying capacitor ...IJECEIAES
This research paper introduces an innovative modulation technique for controlling a 3-level flying capacitor multilevel inverter (FCMLI), aiming to streamline the modulation process in contrast to conventional methods. The proposed
simplified modulation technique paves the way for more straightforward and
efficient control of multilevel inverters, enabling their widespread adoption and
integration into modern power electronic systems. Through the amalgamation of
sinusoidal pulse width modulation (SPWM) with a high-frequency square wave
pulse, this controlling technique attains energy equilibrium across the coupling
capacitor. The modulation scheme incorporates a simplified switching pattern
and a decreased count of voltage references, thereby simplifying the control
algorithm.
This document provides an overview of cloud computing. It begins with Gartner's definition of cloud computing as delivering elastic IT capabilities as a service using internet technologies. The document then discusses the history of cloud computing from John McCarthy's vision in 1961 to modern cloud services from Amazon, Google, and Microsoft. It explores how cloud computing solves the scalability problem by allowing resources to rapidly scale up or down as needed. The document outlines the key cloud service models of IaaS, PaaS, and SaaS and provides examples. It also shares success stories of companies like Zynga and Netflix leveraging cloud computing. Finally, it discusses potential cost savings and benefits of cloud computing through case studies.
Introduction to Cloud Computing and Big Datawaheed751
This document provides an introduction to cloud computing and big data. It defines cloud computing as a model for providing scalable computing resources over the internet with minimal management. The key characteristics of cloud computing include on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service. Dynamic provisioning allows cloud resources to scale up and down based on demand. This helps solve the problems of underutilization and overload in traditional systems with static capacity. The document also discusses how dynamic provisioning can be used in multi-tier web applications running on cloud infrastructure.
Course 3 : Types of data and opportunities by Nikolaos DeligiannisBetacowork
This document discusses big data and opportunities related to different types of data. It covers challenges of big data including volume, velocity, variety and veracity. It also discusses value that can be extracted from data. The document outlines static versus real-time data and structured versus unstructured data. Examples of applying machine learning techniques like regression, classification, clustering and dimensionality reduction are provided. The introduction to cloud computing discusses public, private and hybrid clouds and features of cloud infrastructure.
The document discusses cloud computing, providing definitions and describing its evolution, service models (SaaS, PaaS, IaaS), deployment models (public, private, hybrid), and advantages/disadvantages. It defines cloud computing as on-demand access to shared pools of configurable computing resources via the internet. The three main service models are software as a service (SaaS), platform as a service (PaaS), and infrastructure as a service (IaaS). Public clouds are hosted by third parties, private clouds are operated solely by a single organization, and hybrid clouds combine public and private.
Google not all clouds are created equal - sap sapphire 2014 (1)David Torres
Google Cloud Platform is built using Google's globally connected infrastructure that has been optimized over 15+ years for scalability, performance, and quality. It provides data processing, storage, and analytics services like Compute Engine, BigQuery, and Cloud Storage. Customers can use these services to build and host applications, process vast amounts of data using MapReduce/Hadoop, and perform digital marketing analytics on large datasets.
This document summarizes the hype and growth around cloud computing, listing major cloud providers like AWS, Microsoft Azure, and Google Cloud. It notes that cloud computing is projected to grow from $40.7 billion in 2010 to $241 billion by 2020. The document also categorizes clouds as either public (accessible to any paying customer) or private (only accessible to company employees). It provides examples of how public clouds from AWS and Google charge for services like compute hours and storage. Finally, it highlights how several large companies have significantly reduced their IT costs and improved provisioning times by leveraging cloud computing.
Cloud computing allows consumers and businesses to use applications without installation and access files from any internet-connected device. It provides shared computing resources over a network on-demand as a utility. There are concerns around security and privacy as cloud providers have control over user data. However, new approaches around information-centric security aim to give more control to users. A shift to cloud computing could benefit large internet companies while traditional software producers may face challenges adapting. It remains to be seen if cloud computing will ultimately become the dominant IT model.
This document provides an overview of cloud computing from the perspective of UC Berkeley's Reliable Adaptive Distributed Systems (RAD) Lab. It discusses what is new about cloud computing, challenges and opportunities, considerations for moving workloads to the cloud, and the RAD Lab's experiences using public and private clouds. Some key points:
- Cloud computing enables on-demand access to computing resources without long-term commitments, allowing for flexible scaling up or down. This transfers risk from users to providers.
- Challenges include lock-in, availability, data transfer bottlenecks, and policy issues. Opportunities include risk transfer enabling new scenarios, standardization, and pay-as-you-go licensing models.
This document provides an overview of cloud computing concepts including:
- The key characteristics of cloud computing including on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service.
- The roots of cloud computing in technologies like virtualization, distributed computing, web services, and utility computing.
- The different service models of cloud computing including Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).
This document discusses various topics related to cloud computing including big data and cloud, cloud as a utility, service availability, performance predictability, providing elasticity, data confidentiality, data lock-in, optimizing data placement and transfer, green cloud, emerging opportunities, and more research questions. It notes that while cloud computing provides benefits, there are also open challenges regarding reliability, security, flexibility and optimization that require further research.
Its a complete very concise yet effective presentation on cloud computing. This is an emerging technique in developing countries like african countries.
The document provides an overview of using Amazon Web Services (AWS) for high-performance computing (HPC) clusters. It discusses how AWS enables scientists to build HPC clusters on demand that can scale up and down based on workload needs. Specific solutions and services mentioned include Alces Flight for launching ready-to-compute HPC clusters on AWS in minutes, the AWS Spot Market for accessing spare computing capacity at low costs, and examples of using AWS for scientific workloads like satellite image analysis and computational fluid design simulations.
This document provides an introduction and overview of cloud computing. It defines cloud computing as applications and services delivered over the internet, as well as the hardware and systems that provide those services. The document discusses common cloud services like SaaS, utility computing using Amazon EC2 as an example, and the key characteristics of cloud computing. It also covers challenges of cloud computing like availability, data security, and performance unpredictability, as well as the growth and future potential of cloud computing.
cloud computing - concepts and technologies and mechanisms of tackling problems in cloud
you plz ignore who created it , plz focus on problem oriented points
This document discusses cloud computing concepts, technologies, and business implications. It provides an introduction to cloud models like IaaS, PaaS, and SaaS and demonstrates cloud capabilities through examples of Amazon AWS, Google App Engine, and Windows Azure. The document also discusses enabling technologies for cloud computing like virtualization and programming models for big data like MapReduce and Hadoop.
Cloud computing provides on-demand access to shared computing resources like networks, servers, storage, applications and services over the internet. It has seen rapid growth in recent years. There are different service models like Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS) depending on what capabilities are provided to the user. Cloud computing can be deployed using private, public, hybrid or community models depending on who manages the infrastructure and who has access to it. While cloud computing provides benefits like flexibility, scalability and cost savings, concerns around security, privacy and reliability remain challenges to adoption.
An improved modulation technique suitable for a three level flying capacitor ...IJECEIAES
This research paper introduces an innovative modulation technique for controlling a 3-level flying capacitor multilevel inverter (FCMLI), aiming to streamline the modulation process in contrast to conventional methods. The proposed
simplified modulation technique paves the way for more straightforward and
efficient control of multilevel inverters, enabling their widespread adoption and
integration into modern power electronic systems. Through the amalgamation of
sinusoidal pulse width modulation (SPWM) with a high-frequency square wave
pulse, this controlling technique attains energy equilibrium across the coupling
capacitor. The modulation scheme incorporates a simplified switching pattern
and a decreased count of voltage references, thereby simplifying the control
algorithm.
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...IJECEIAES
Climate change's impact on the planet forced the United Nations and governments to promote green energies and electric transportation. The deployments of photovoltaic (PV) and electric vehicle (EV) systems gained stronger momentum due to their numerous advantages over fossil fuel types. The advantages go beyond sustainability to reach financial support and stability. The work in this paper introduces the hybrid system between PV and EV to support industrial and commercial plants. This paper covers the theoretical framework of the proposed hybrid system including the required equation to complete the cost analysis when PV and EV are present. In addition, the proposed design diagram which sets the priorities and requirements of the system is presented. The proposed approach allows setup to advance their power stability, especially during power outages. The presented information supports researchers and plant owners to complete the necessary analysis while promoting the deployment of clean energy. The result of a case study that represents a dairy milk farmer supports the theoretical works and highlights its advanced benefits to existing plants. The short return on investment of the proposed approach supports the paper's novelty approach for the sustainable electrical system. In addition, the proposed system allows for an isolated power setup without the need for a transmission line which enhances the safety of the electrical network
Null Bangalore | Pentesters Approach to AWS IAMDivyanshu
#Abstract:
- Learn more about the real-world methods for auditing AWS IAM (Identity and Access Management) as a pentester. So let us proceed with a brief discussion of IAM as well as some typical misconfigurations and their potential exploits in order to reinforce the understanding of IAM security best practices.
- Gain actionable insights into AWS IAM policies and roles, using hands on approach.
#Prerequisites:
- Basic understanding of AWS services and architecture
- Familiarity with cloud security concepts
- Experience using the AWS Management Console or AWS CLI.
- For hands on lab create account on [killercoda.com](https://killercoda.com/cloudsecurity-scenario/)
# Scenario Covered:
- Basics of IAM in AWS
- Implementing IAM Policies with Least Privilege to Manage S3 Bucket
- Objective: Create an S3 bucket with least privilege IAM policy and validate access.
- Steps:
- Create S3 bucket.
- Attach least privilege policy to IAM user.
- Validate access.
- Exploiting IAM PassRole Misconfiguration
-Allows a user to pass a specific IAM role to an AWS service (ec2), typically used for service access delegation. Then exploit PassRole Misconfiguration granting unauthorized access to sensitive resources.
- Objective: Demonstrate how a PassRole misconfiguration can grant unauthorized access.
- Steps:
- Allow user to pass IAM role to EC2.
- Exploit misconfiguration for unauthorized access.
- Access sensitive resources.
- Exploiting IAM AssumeRole Misconfiguration with Overly Permissive Role
- An overly permissive IAM role configuration can lead to privilege escalation by creating a role with administrative privileges and allow a user to assume this role.
- Objective: Show how overly permissive IAM roles can lead to privilege escalation.
- Steps:
- Create role with administrative privileges.
- Allow user to assume the role.
- Perform administrative actions.
- Differentiation between PassRole vs AssumeRole
Try at [killercoda.com](https://killercoda.com/cloudsecurity-scenario/)
Advanced control scheme of doubly fed induction generator for wind turbine us...IJECEIAES
This paper describes a speed control device for generating electrical energy on an electricity network based on the doubly fed induction generator (DFIG) used for wind power conversion systems. At first, a double-fed induction generator model was constructed. A control law is formulated to govern the flow of energy between the stator of a DFIG and the energy network using three types of controllers: proportional integral (PI), sliding mode controller (SMC) and second order sliding mode controller (SOSMC). Their different results in terms of power reference tracking, reaction to unexpected speed fluctuations, sensitivity to perturbations, and resilience against machine parameter alterations are compared. MATLAB/Simulink was used to conduct the simulations for the preceding study. Multiple simulations have shown very satisfying results, and the investigations demonstrate the efficacy and power-enhancing capabilities of the suggested control system.
artificial intelligence and data science contents.pptxGauravCar
What is artificial intelligence? Artificial intelligence is the ability of a computer or computer-controlled robot to perform tasks that are commonly associated with the intellectual processes characteristic of humans, such as the ability to reason.
› ...
Artificial intelligence (AI) | Definitio
3. The Hype!
• Forrester in 2010 – Cloud computing will go from
$40.7 billion in 2010 to $241 billion in 2020.
• Goldman Sachs says cloud computing will grow
at annual rate of 30% from 2013-2018
• Hadoop market to reach $20.8 B by by 2018:
Transparency Market Research
• Companies and even Federal/state governments
using cloud computing now: fbo.gov
3
4. Many Cloud Providers
• AWS: Amazon Web Services
– EC2: Elastic Compute Cloud
– S3: Simple Storage Service
– EBS: Elastic Block Storage
• Microsoft Azure
• Google Cloud/Compute Engine/AppEngine
• Rightscale, Salesforce, EMC, Gigaspaces, 10gen, Datastax,
Oracle, VMWare, Yahoo, Cloudera
• And many many more!
4
5. Two Categories of Clouds
• Can be either a (i) public cloud, or (ii) private cloud
• Private clouds are accessible only to company employees
• Public clouds provide service to any paying customer:
– Amazon S3 (Simple Storage Service): store arbitrary datasets, pay per GB-month
stored
• As of 2019: 0.4c-3 c per GB month
– Amazon EC2 (Elastic Compute Cloud): upload and run arbitrary OS images, pay
per CPU hour used
• As of 2019: 0.2 c per CPU hr to $7.2 per CPU hr (depending on strength)
– Google cloud: similar pricing as above
– Google AppEngine/Compute Engine: develop applications within their appengine
framework, upload data that will be imported into their format, and run
5
6. Customers Save Time and $$$
• Dave Power, Associate Information Consultant at Eli Lilly and
Company: “With AWS, Powers said, a new server can be up and
running in three minutes (it used to take Eli Lilly seven and a half
weeks to deploy a server internally) and a 64-node Linux cluster can
be online in five minutes (compared with three months internally). …
It's just shy of instantaneous.”
• Ingo Elfering, Vice President of Information Technology Strategy,
GlaxoSmithKline: “With Online Services, we are able to reduce our IT
operational costs by roughly 30% of what we’re spending”
• Jim Swartz, CIO, Sybase: “At Sybase, a private cloud of virtual servers
inside its datacenter has saved nearly $US2 million annually since
2006, Swartz says, because the company can share computing power
and storage resources across servers.”
• 100s of startups in Silicon Valley can harness large computing resources
without buying their own machines.
6
8. What is a Cloud?
• It’s a cluster!
• It’s a supercomputer!
• It’s a datastore!
• It’s superman!
• None of the above
• All of the above
• Cloud = Lots of storage + compute cycles nearby
8
9. What is a Cloud?
• A single-site cloud (aka “Datacenter”) consists
of
– Compute nodes (grouped into racks) (2)
– Switches, connecting the racks
– A network topology, e.g., hierarchical
– Storage (backend) nodes connected to the network
(3)
– Front-end for submitting jobs and receiving client
requests (1)
– (1-3: Often called “three-tier architecture”)
– Software Services
• A geographically distributed cloud consists of
– Multiple such sites
– Each site perhaps with a different structure and
services
9
11. “A Cloudy History of Time”
1940
1950
1960
1970
1980
1990
2000
Timesharing Companies
& Data Processing Industry
Grids
Peer to peer systems
Clusters
The first datacenters!
PCs
(not distributed!)
Clouds and datacenters
2012
11
12. “A Cloudy History of Time”
1940
1950
1960
1970
1980
1990
2000
2012 Clouds
Grids (1980s-2000s):
•GriPhyN (1970s-80s)
•Open Science Grid and Lambda Rail (2000s)
•Globus & other standards (1990s-2000s)
Timesharing Industry (1975):
•Market Share: Honeywell 34%, IBM 15%,
•Xerox 10%, CDC 10%, DEC 10%, UNIVAC 10%
•Honeywell 6000 & 635, IBM 370/168,
Xerox 940 & Sigma 9, DEC PDP-10, UNIVAC 1108
Data Processing Industry
- 1968: $70 M. 1978: $3.15 Billion
First large datacenters: ENIAC, ORDVAC, ILLIAC
Many used vacuum tubes and mechanical relays
Berkeley NOW Project
Supercomputers
Server Farms (e.g., Oceano)
P2P Systems (90s-00s)
•Many Millions of users
•Many GB per day
12
13. Trends: Technology
• Doubling Periods – storage: 12 mos, bandwidth: 9 mos,
and (what law is this?) cpu compute capacity: 18 mos
• Then and Now
– Bandwidth
• 1985: mostly 56Kbps links nationwide
• 2015: Tbps links widespread
– Disk capacity
• Today’s PCs have TBs, far more than a 1990 supercomputer
13
14. Trends: Users
• Then and Now
Biologists:
– 1990: were running small single-molecule
simulations
– Today: CERN’s Large Hadron Collider producing
many PB/year
14
15. Prophecies
• In 1965, MIT's Fernando Corbató and the other designers of
the Multics operating system envisioned a computer facility
operating “like a power company or water company”.
• Plug your thin client into the computing Utility and Play
your favorite Intensive Compute & Communicate
Application
– Have today’s clouds brought us closer to this reality? Think
about it.
15
16. Four Features New in Today’s
Clouds
I. Massive scale.
II. On-demand access: Pay-as-you-go, no upfront commitment.
– And anyone can access it
III. Data-intensive Nature: What was MBs has now become TBs, PBs and
XBs.
– Daily logs, forensics, Web data, etc.
– Humans have data numbness: Wikipedia (large) compressed is only about 10 GB!
IV. New Cloud Programming Paradigms: MapReduce/Hadoop,
NoSQL/Cassandra/MongoDB and many others.
– High in accessibility and ease of programmability
– Lots of open-source
Combination of one or more of these gives rise to novel and unsolved
distributed computing problems in cloud computing. 16
17. I. Massive Scale
• Facebook [GigaOm, 2012]
– 30K in 2009 -> 60K in 2010 -> 180K in 2012
• Microsoft [NYTimes, 2008]
– 150K machines
– Growth rate of 10K per month
– 80K total running Bing
– In 2013, Microsoft Cosmos had 110K machines (4 sites)
• Yahoo! [2009]:
– 100K
– Split into clusters of 4000
• AWS EC2 [Randy Bias, 2009]
– 40K machines
– 8 cores/machine
• eBay [2012]: 50K machines
• HP [2012]: 380K in 180 DCs
• Google [2011, Data Center Knowledge] : 900K
17
19. Quiz: Where is the World’s
Largest Datacenter?
• (2018) China Telecom. 10.7 Million sq. ft.
• (2017) “The Citadel” Nevada. 7.2 Million sq. ft.
• (2015) In Chicago!
• 350 East Cermak, Chicago, 1.1 MILLION sq. ft.
• Shared by many different “carriers”
• Critical to Chicago Mercantile Exchange
• See:
– https://www.gigabitmagazine.com/top10/top-10-biggest-data-centres-world
– https://www.racksolutions.com/news/data-center-news/top-10-largest-data-centers-world/
19
20. What does a datacenter look like
from inside?
• A virtual walk through a datacenter
• Reference: http://gigaom.com/cleantech/a-rare-look-
inside-facebooks-oregon-data-center-photos-video/
20
22. Power
Off-site
On-site
•WUE = Annual Water Usage / IT Equipment Energy (L/kWh) – low is good
•PUE = Total facility Power / IT Equipment Power – low is good
(e.g., Google~1.1)
22
23. Cooling
Air sucked in from top (also, Bugzappers) Water purified
Water sprayed into air 15 motors per server bank
23
24. Extra - Fun Videos to Watch
• Microsoft GFS Datacenter Tour (Youtube)
– http://www.youtube.com/watch?v=hOxA1l1pQIw
• Timelapse of a Datacenter Construction on the Inside
(Fortune 500 company)
– http://www.youtube.com/watch?v=ujO-xNvXj3g
24
25. II. On-demand access: *aaS
Classification
On-demand: renting a cab vs. (previously) renting a car, or buying one. E.g.:
– AWS Elastic Compute Cloud (EC2): a few cents to a few $ per CPU hour
– AWS Simple Storage Service (S3): a few cents per GB-month
• HaaS: Hardware as a Service
– You get access to barebones hardware machines, do whatever you want with them,
Ex: Your own cluster
– Not always a good idea because of security risks
• IaaS: Infrastructure as a Service
– You get access to flexible computing and storage infrastructure. Virtualization is one
way of achieving this (cgroups, Kubernetes, Dockers, VMs,…). Often said to
subsume HaaS.
– Ex: Amazon Web Services (AWS: EC2 and S3), OpenStack, Eucalyptus, Rightscale,
Microsoft Azure, Google Cloud. 25
26. II. On-demand access: *aaS
Classification
• PaaS: Platform as a Service
– You get access to flexible computing and storage infrastructure,
coupled with a software platform (often tightly coupled)
– Ex: Google’s AppEngine (Python, Java, Go)
• SaaS: Software as a Service
– You get access to software services, when you need them. Often
said to subsume SOA (Service Oriented Architectures).
– Ex: Google docs, MS Office 365 Online
26
27. III. Data-intensive Computing
• Computation-Intensive Computing
– Example areas: MPI-based, High-performance computing, Grids
– Typically run on supercomputers (e.g., NCSA Blue Waters)
• Data-Intensive
– Typically store data at datacenters
– Use compute nodes nearby
– Compute nodes run computation services
• In data-intensive computing, the focus shifts from computation to the data:
CPU utilization no longer the most important resource metric, instead I/O is
(disk and/or network)
27
28. IV. New Cloud Programming
Paradigms
• Easy to write and run highly parallel programs in new cloud programming
paradigms:
– Google: MapReduce and Sawzall
– Amazon: Elastic MapReduce service (pay-as-you-go)
– Google (MapReduce)
• Indexing: a chain of 24 MapReduce jobs
• ~200K jobs processing 50PB/month (in 2006)
– Yahoo! (Hadoop + Pig)
• WebMap: a chain of several MapReduce jobs
• 300 TB of data, 10K cores, many tens of hours (~2008)
– Facebook (Hadoop + Hive)
• ~300TB total, adding 2TB/day (in 2008)
• 3K jobs processing 55TB/day
– Similar numbers from other companies, e.g., Yieldex, eharmony.com, etc.
– NoSQL: MySQL is an industry standard, but Cassandra is 2400 times faster! 28
29. Two Categories of Clouds
• Can be either a (i) public cloud, or (ii) private cloud
• Private clouds are accessible only to company employees
• Public clouds provide service to any paying customer
• You’re starting a new service/company: should you use a public cloud
or purchase your own private cloud?
29
30. Single site Cloud: to Outsource
or Own?
• Medium-sized organization: wishes to run a service for M months
– Service requires 128 servers (1024 cores) and 524 TB
– Same as UIUC CCT (Cloud Computing Testbed) cloud site (bought in 2009, now
decommissioned)
• Outsource (e.g., via AWS): monthly cost
– S3 costs: $0.12 per GB month. EC2 costs: $0.10 per CPU hour (costs from 2009)
– Storage = $ 0.12 X 524 X 1000 ~ $62 K
– Total = Storage + CPUs = $62 K + $0.10 X 1024 X 24 X 30 ~ $136 K
• Own: monthly cost
– Storage ~ $349 K / M
– Total ~ $ 1555 K / M + 7.5 K (includes 1 sysadmin / 100 nodes)
• using 0.45:0.4:0.15 split for hardware:power:network and 3 year
lifetime of hardware 30
31. Single site Cloud: to Outsource
or Own?
• Breakeven analysis: more preferable to own if:
- $349 K / M < $62 K (storage)
- $ 1555 K / M + 7.5 K < $136 K (overall)
Breakeven points
- M > 5.55 months (storage)
- M > 12 months (overall)
- As a result
- Startups use clouds a lot
- Cloud providers benefit monetarily most from storage
31
34. Public Research Clouds
• Accessible to researchers with a qualifying grant
• Chameleon Cloud: https://www.chameleoncloud.org/
• HaaS
• OpenStack (~AWS)
• CloudLab: https://www.cloudlab.us/
• Build your own cloud on their hardware
34
35. Summary
• Clouds build on many previous generations of distributed
systems
• Especially the timesharing and data processing industry of
the 1960-70s.
• Need to identify unique aspects of a problem to classify it as
a new cloud computing problem
– Scale, On-demand access, data-intensive, new programming
• Otherwise, the solutions to your problem may already exist!
• Next: Mapreduce!
35