"The Cloud"Different meaning: (a) user perspective, (b) software engineer /systems perspectivesource: wikipedia.org source: shanghaiwebhosting.com
Cloud Computing● The natural evolution of widely used technologies and computing paradigms, like: ○ the client-server model ○ virtualization (dynamic multiple "virtual" machines contained into a physical one) ○ the service oriented architecture ○ utility computing● The "Cloud Computing" buzzword focuses more on the business model than on the technologies involved ○ computing/network resources consumed on demand ○ elasticity - scalability ○ service, resources managed by the provider
Type of Cloud Computing Services● Based on the service offered: ○ Infrastructure-as-a-Service (IaaS) - Amazon AWS ○ Platform-as-a-Service (PaaS) - Google Appengine ○ may also include Software-as-a-Service (SaaS) - SalesForce.com● Based on ownership: ○ Public clouds - offer services to everyone ○ Private clouds - in-house solutions ○ Hybrid clouds - little bit of both
the goal of cloud computing is to provide on demand, scalable access to computing resources & IT services
How ? IaaS: you built your software and rent ○ Virtual machines (time based) ○ Storage (e.g. bytes per month) ○ Traffic in/out (per month) ○ Additional services (via API calls) ○ Deploy your software components and you manage elasticity & scalabilityPaaS: ○ Deploy your specially designed application and the service handles elastic usage of resources / scalingSaaS: you rent the usage of a software product ○ on a subscription basis (per user cost) ○ or on demand (time & per user cost) ○ some application customization is offered
What, how, why in 3 bullets... ● Whats Amazon Web Services: pay-as-you-go access to cloud infrastructure and elastic services via REST/SOAP and/or a Web-based console. ● How (simplified): ○ create machine images (AMIs) ○ launch VMs (boot from AMIs) ○ attach persistent volumes (EBS vols) ○ login and install software & applications ○ (optional) use other AWS services via REST/SOAP ● Why: ○ dynamic, elastic usage of resources ○ cost-effective high availability (e.g. S3) ○ pay on usage (hourly-rate)
AWS products & services● EC2: your virtual machines in predefined HW configurations● EBS: persistent block storage (faster than S3, slower than volatile EC2 instance disks)● S3: object (file) storage via REST/SOAP, high availability and geographical replication, also used for your AMIs● CloudFront: Amazons CDN● Cloudwatch / Autoscaling: monitoring and scaling of EC2● Elastic Cache: distributed in-memory cache
AWS products & services (contd)● SQS: distributed queue● SNS: notification service(messaging / topics & events)● SES: scalable, bulk e-mail service● SimpleDB / RDS: Amazons NoSQL / RDBMS services● Elastic MapReduce: Hadoop on EC2/S3● and more on deployment and management, payments/e- commerce, networking... etc
Costs● There is a free usage tier: new user sign-up, one year for a Linux micro instance● Pay as you go for what you use ○ pay usage per hour ○ pay for traffic (volume) ○ pay for storage (volume) ○ pay for I/O operations / transactions● Much more expensive than the typical rented physical machines / VMs
Expensive? Why should I use it?● Cost effective high availability● Elasticity, easy to scale up/down dynamically● REST-based infrastructure management & monitoring● Massive processing power for short periods ○ e.g. launch 10.000 VMs for a week ○ MapReduce big data projects● Its not cost-effective to use EC2 instances for your low- traffic web site (maybe on the free-tier)
● SaaS offering implemented by EBS.gr● A distributed, scalable and elastic file storage platform that: ○ supports access via multiple user interfaces (web browser, mobile devices, desktop, WebDAV) ○ provides an API for building custom client applications ○ it is based on EBSs open source project gss-project.org● It is designed for operating in the cloud, currently deployed at the Amazon cloud.
High level architecture requirements● MyNetworkFolders requires multiple servers for hosting its software components: ○ duplicated (for fault tolerance) front-end web server ○ cluster of (at least two) application servers implementing the core business logic ○ database cluster for storing meta-data and other info ○ search / indexing server cluster for full text search ○ a secure, efficient and scalable file storage system● All these add up to at least three virtual servers and can easily grow to clusters of tens of servers.
Hosted vs the Cloud● Old school hosting solution: private or rented physical servers in a data center: ○ always maintain and support a minimum number of servers even when the load is low ○ energy & cost efficient adaptation to fluctuating load is almost impossible to achieve ○ scaling up requires buying/renting and setting up new physical servers (and supporting equipment) ■ for the application ■ for the storage subsystem ○ high operating costs for a small company, especially for achieving a high level of fault tolerance / availability
Hosted vs the Cloud● Having mynetworkfolders.com hosted in the cloud: ○ use and pay only what we need ○ easily and dynamically grow or shrink resource usage (and energy consumption) to meet fluctuating user demand during the day ○ easily scale up (more virtual servers) when the user base increases, or when new functionality is added ○ economy of scale offers high availability and fault tolerance in an affordable cost ○ minimize operational costs
AWS Technologies & Products Used● Currently using in production: ○ EC2 for server instances (app, db, search,admin) ○ S3 - reliable, replicated file storage ■ the main reason we used AWS ○ CloudWatch (monitoring)● Plan to use: ○ Elastic Load Balancing ○ CloudFront