Introduction to Amazon Web Services

1,830 views

Published on

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,830
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
1
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Introduction to Amazon Web Services

  1. 1. Introduction toAmazon Web Services Dayanand Shanmugham http://www.linkedin.com/in/dayanandshanmugham 14 September 2012
  2. 2. Agenda1. What is Amazon Web Services?2. IT & Architecture Perspective: a. Amazon Product Stack b. S3 (Simple Storage Service) Basic Concepts c. Demo of how to use S3 d. Potential Use Cases3. Cloud IT Eco-System / Business Perspective : a. Cloud Storage Costs b. Cloud Service Providers (CSPs) c. Comparison Study Report
  3. 3. Amazon Web Services http://aws.amazon.com/1. Amazon Web Services offers a complete set of infrastructure and application services that enable us to run virtually everything in the cloud – from enterprise applications and big data projects to social games and mobile apps.2. In 2006, Amazon Web Services (AWS) began offering IT infrastructure services to businesses in the form of web services – now commonly known as cloud computing.3. Key benefits is opportunity to replace up-front capital infrastructure expenses with low variable costs that scale with our business: a. Low Cost - Pay-as-you-go pricing b. Agility and Instant Elasticity - Massive global cloud infrastructure c. Open and Flexible - Language and operating system agnostic platform d. Secure - PCI DSS Level 1, ISO 27001, FISMA Moderate, HIPAA & SAS 70 Type II
  4. 4. Amazon Product Stack http://aws.amazon.com/products/ Our Applications
  5. 5. Amazon Global Infrastructure Where are Amazon Web Services available?
  6. 6. Amazon Global Infrastructure 7 Regions – Each region has multiple Availability Zones
  7. 7. Amazon S3 (Simple Storage Service) What is S3? Features? Concepts? Controls? Demo Quote “…Subscriptions to Cloud Storage Services to Reach Half-Billion Level This Year… jump to 625 million next year… projected to hit 1.3 billion in 2017…” http://www.bloomberg.com/article/2012-09-06/aUXSunqkHUP0.htmlLarge organizations such as Amazon, NASA, Netflix, Google, NBC, Zynga, etc depend on Cloud Storage
  8. 8. Amazon S3 (Simple Storage Service) http://aws.amazon.com/s3/ http://[BucketName].s3.amazonaws.com/[Filename]Amazon Simple Storage Service (Amazon S3) is a web service that enables us tostore data in the cloud. We can then download data or use the data with otherAWS services, such as Amazon Elastic Cloud Compute (EC2).Features:• Storage as a Service: through Web Service API• Highly Scalable (Unlimited Storage Space)• High Data Durability (99.999999999%)• Highly Available (99.99%)• Server Side Encryption (Data at Rest)• VersioningS3 Basic Concepts: Object Buckets With Region - Geographical location where Buckets Amazon S3 service is available Objects(Storage Container) 1. Fundamental Entity in Buckets 2. Each Object has a unique Key
  9. 9. Access Controls1. Amazon S3 enables us to manage access to objects and buckets using – we can use them independently or together: – Access control lists (ACLs) – Bucket policies – IAM policies2. ACLs only grant permissions; they do not deny them. ACLs can contain the following grantee types: – Specific AWS accounts – All AWS accounts – Any anonymous request3. Bucket policies provide access control management at the bucket level for – these are collection of JSON statements: – Bucket – Objects4. AWS Identity and Access Management (IAM) enables us to create multiple users within our AWS account and manage their permissions via IAM policies: – Bucket policies are attached to a bucket – IAM policies are attached to individual users in your account
  10. 10. Access Controls http://docs.amazonwebservices.com/AmazonS3/latest/dev/UsingIAMPolicies.htmlExample#1: IAM policy and Bucket Policy gives Bob & Susan permission to Example#2: IAM policy gives Bob permission to use PutObject on access PutObject on bucket_xyz bucket_xyz & bucket policy gives permission to use ListBucket Example#3: Explicit deny always overrides an allow Example#4: Bucket policy denies upload object (s3:PutObject) permission to everyone if the request does not include the x-amz-server-side- encryption header requesting server-side encryption
  11. 11. Basic Operations https://console.aws.amazon.com http://[BucketName].s3.amazonaws.com/[Filename] Store /Write DC2 Read DC1Our App Delete DC3 List Demo of S3 using Amazon Web Service Console
  12. 12. Distributed Storage Systems StrategyIf we have to design S3? Design Challenges? How?
  13. 13. Distributed Storage Systems StrategyOption#1 Store /Write DC2 Allow Replication To DC1 Our App Complete DC3 Response (OK) List
  14. 14. Distributed Storage Systems StrategyOption#2 Store /Write DC2 Response (OK) DC1 Our App Replicate after Data is stored DC3 in DC1 List
  15. 15. Behaviour of S3 inConcurrent Read(s) & Write(s) http://[BucketName].s3.amazonaws.com/[Filename]
  16. 16. Potential Use Cases / Case StudiesS3 in Overall IT Application Architecture – When to use? Need?
  17. 17. Potential Use Cases / Case StudiesNeed:1. This enterprise manages two disparate sets of information. Table-oriented data is maintained in an on-premise Oracle database, while a SAN is used as a repository for file-based information.2. For further safeguarding of these vital assets, tapes are used for backup and disaster recovery purposes.3. Approximately 20 GB of new information is generated each day.4. Unfortunately, the backup and archive management processes are cumbersome and expensive, while restoring archived information can take days to completeSolution:1. Architects decide to use AWS cloud2. For file-based artifacts, access to Amazon S3 will be via the Amazon AWS SDK for Java and AWS Toolkit for Eclipse3. Storage architects will create one or more uniquely- identified buckets in Amazon S3, each of which can hold an unlimited amount of backup data4. For relational data, Oracle’s Secure Backup Cloud Module will be able to take advantage of existing RMAN scripts to back up information directly from the Oracle database into Amazon S35. Third-party storage management solution can be used to manage the entire process, including encryption and other security details
  18. 18. Potential Use Cases / Case StudiesNeed:1. This company provides a tremendous number of multimedia files (audio files, videos, and images), which are stored on internally hosted servers and made available over the Internet. In addition to the multimedia files themselves, each file requires a significant amount of metadata (such as title, author, keywords, size, and so on).2. On the plus side, their web site is a hit3. Unfortunately, internal servers are failing to keep up with this demand — the sheer amount of data will soon overwhelm available disk storage, and the amount of necessary metadata indexing is outstripping processing capacitySolution:1. Architects decide to publish all content to the AWS cloud-based storage, thereby eliminating the need to purchase and maintain internal servers.2. Users will directly fetch content from AWS-based storage
  19. 19. Potential Use Cases / Case StudiesNeed:1. A large corporation is maintaining a data warehouse on a High-CPU Extra Large Amazon EC2 instance, with ten 800 GB EBS volumes holding the information itself2. While this architecture successfully meets business needs, some new requirements will mandate an extension to their storage composition. Specifically, a change to organizational policy now requires frequent data snapshots; these images need to archive supplementary data approximately every one to two hours.Solution:1. Architects decide to add Amazon S3 to the mixture2. Amazon EC2 instance and supporting EBS volumes will continue in their current roles3. Developers will write a script or small application that uses Amazon EBS API to create incremental snapshots stored in Amazon S34. The new snapshot application will be run every 120 minutes using a Linux cron job or a Windows scheduled task
  20. 20. Potential Use Cases / Case StudiesNeed:1. A New SaaS application is in the planning stages2. Solution will need to manage a variety of information, including traditional relational data, frequently changing status feeds, and large amounts of multimedia.3. The logic behind the application will be complex and processing- intensive, needing numerous joins of relational data to produce the required results.4. Finally, as is the case with virtually all of these types of solutions, users will demand high availability and fault tolerance.5. Rather than being forced to hand-code all of these availability- oriented capabilities, the designers hope to leverage the cloud for features such as replication, scalability, and automated backups, freeing their time for developing the core application logic.Solution:1. The Architects of the new SaaS package opt for a collection of AWS storage offerings, each of which will handle a specific responsibility.2. RDS will serve as the repository for all information that requires full relational database infrastructure3. To process complex cross-table joins, RDS will deliver high availability via its well-proven data replication architecture4. Read operations will be served by the slaves, with writes being processed on the master—all of which will be automatically managed by RDS5. RDS will provide database administration as a service, with one-click vertical scaling, elastic storage, and automated backups6. Multimedia objects will reside in Amazon S3, while SimpleDB will be tasked with maintaining the continually updated status feeds
  21. 21. Potential Use Cases / Case StudiesNeed:1. This web-based social application has seen a massive spike in traffic - likely cause for this momentum is the release of a new client application designed for Smartphones2. Server-side information repository was originally deployed using an on-premise MySQL database3. No longer possible to support the number of clients or the amount of managed information with that database aloneSolution:1. Architects want to avoid forcing a costly and time-consuming rewrite of their primary application, so most of the database-focused logic must remain unchanged
  22. 22. Cloud IT Eco-System Costs? Is Cloud Storage economical?
  23. 23. “File Storage Costs Less In The Cloud Than In-House” By Forrestor http://media.amazonwebservices.com/Forrester_File_Storage_Costs_Less_In_The_Cloud.pdf http://aws.amazon.com/s3/#pricing
  24. 24. “File Storage Costs Less In The Cloud Than In-House” By Forrestor http://media.amazonwebservices.com/Forrester_File_Storage_Costs_Less_In_The_Cloud.pdf http://aws.amazon.com/s3/#pricing
  25. 25. “File Storage Costs Less In The Cloud Than In-House” By Forrestor http://media.amazonwebservices.com/Forrester_File_Storage_Costs_Less_In_The_Cloud.pdf http://aws.amazon.com/s3/#pricing
  26. 26. Cloud IT Eco-System Other Cloud Service Providers?
  27. 27. Cloud IT Eco-System Cloud Storage Providers 1. Amazon Simple Storage Service (S3) 2. AT&T Synaptic Storage as a Service 3. Box.net 4. Choopa Cloud Object Storage 5. Cloud Drive 6. CloudOne Storage 7. Egnyte HybridCloud 8. Enzu Cloud Storage 9. GoGrid Cloud Storage 10. Iron Mountain Storage-as-a-Service 11. Layered Tech Depot 12. Mozzy 13. Nasuni 14. Netcetera vStorage 15. NetDepot Cloud Storage 16. OpSource Cloud Files 17. Rackspace Cloud Files 18. Softlayer Cloudlayer Storage 19. Storage Qloud 20. SugarSync 21. Syncplicity Virtual Private Cloud 22. The Planet Storage Cloud 23. Windows Azure Storage Service 24. Zetta Storage Service 25. …
  28. 28. Cloud IT Eco-System Content Delivery Networks 1. Akamai Media Delivery 21. Level 3 CDN 2. Amazon Cloudfront 22. Limelight Networks 3. BitGravity 23. LocalMirror 4. Cachefly 24. MaxCDN 5. CDN Video 25. Mirror Image 6. CDN77 26. NetDNA 7. CDNetworks 27. Ngenix 8. ChinaCache 28. Prime Networks 9. ChinaNetCenter CDN 29. Rapidedge CDN 10. CloudCache 30. SimpleCDN 11. CloudFlare CDN 31. Softlayer Cloudlayer CDN 12. Cotendo 32. Tata Communications CDN 13. EdgeCast Networks 33. UPX CloudCache 14. Fastly 34. Value CDN 15. Fastweb CDN 35. VoxCAST 16. GoGrid CDN 36. VPS.NET CDN 17. Highwinds 37. … 18. Hostway CDN 19. Internap CDN 20. Internode CDN
  29. 29. Cloud IT Eco-System Cloud Hosting Providers1. 1&1 Dynamic Cloud Server 31. iland 61. The Planet Server Cloud2. Atlantic Cloud Servers 32. Instance Cloud Computing 62. VI Cloud Hosting3. BlueLock 33. iomart Cloud Hosting 63. VoxCLOUD4. CariNet 34. iWeb Smart Servers 64. VPS.NET5. Cartika Cloud 35. Joyent Cloud 65. VPSLand Bare Metal Cloud Servers6. Cloud Central 36. Layered Tech Virtual Machines 66. WestHost Cloud Hosting7. Cloud.bg 37. Logicworks Cloud Computing 67. Windows Azure Compute Service8. CloudSigma 38. Mediatemple 68. Zerigo Cloud Servers9. Combell Cloud Servers 39. NaviSite Managed Cloud Services 69. Amazon Elastic Compute Cloud (EC2)10. Connectria Cloud Hosting 40. NetDepot Cloud Servers 70. AT&T Synaptic Compute as a Service11. Crackerhost 41. NewServers 71. Go Daddy Cloud Server (Mac OS X)12. Datapipe Managed Cloud 42. Openhosting vServer 72. AirVM13. Dediserve Cloud Machines 43. OpSource Cloud Servers 73. …14. DigitalOcean 44. OrcsWeb Windows Cloud Server15. DotBlock Cloud VPS 45. OVH Cloud16. ElasticHosts 46. Peak 10 Cloud Services17. Engine Yard AppCloud 47. Rackforce ddsCloud18. Enzu Cloud Hosting 48. Rackspace Cloud Servers19. FireHost 49. Razor Servers Cloud20. FlexiScale 50. ReliaCloud21. Gandi Cloud VPS 51. Server4You vServer Cloud22. Gigenetcloud 52. ServerCentral Cloud Services23. GoGrid Cloud Servers 53. SingleHop Cascade24. Google App Engine 54. SiteCloud25. Hivelocity PDC 55. Slicehost26. Hostbasket Dynamic Cloud 56. Softlayer Cloudlayer Computing27. HostIcan VPS 57. Storm On Demand28. Hosting.com 58. SunGard Cloud Platform29. Hostway Cloud Servers 59. Tektonic Cloud VPS30. Hpchost Cloud Hosting 60. Terremark vCloud Express
  30. 30. Cloud IT Eco-System Which Cloud Provider(s) to use?
  31. 31. Comparative Study (Done by Nasuni) http://www.nasuni.com/downloads/resources/87/the_state_of_cloud_storage.pdfBackground:1. In April 2009, Nasuni began an ongoing and unprecedented evaluation process of “Cloud Service Providers (CSPs) based on a wide array of factors2. Technical test looked at how CSPs perform in a specific use case, namely, how they perform for organizations that want to take advantage of the cloud for primary storage, data protection and disaster recovery. Nasuni did not test for every possible use case.3. Test focus on three key areas of importance for organizations that want to take advantage of cloud storage: – Performance: The cloud needs to respond quickly to queries and not slow when stressed. If performance is poor, organizations will spend too much time waiting on the cloud and productivity will suffer. – Stability/Availability: If organizations are to trust critical data to the cloud, it must be available at all times. – Scalability: One of the primary advantages of cloud storage is unlimited capacity. Without this property, the cloud is much less valuable to organizations.
  32. 32. Comparative Study Results http://www.nasuni.com/downloads/resources/87/the_state_of_cloud_storage.pdfResults:1. Ultimately, only 6 of 16 providers passed Nasuni’s testing: – Amazon S3 – AT&T Synaptic Storage as a Service – Microsoft Azure – Nirvanix – Peer1 Hosting – Rackspace Cloud2. Only Two Cloud Service Providers emerged as top performers in the Nasuni study: – Amazon S3 – Microsoft Azure (Note: Amazon S3 standout across all evaluation areas)
  33. 33. Comparative Study Methodology http://www.nasuni.com/downloads/resources/87/the_state_of_cloud_storage.pdfMethodology:1. API Integration – To ensure that it is possible to test the service at all2. Unit Testing – Larger software components are broken down into their building blocks (units) and then tested for inputs, outputs and error cases3. Performance Testing – To measure response time (how quickly one can interact with the cloud), throughput (how fast data can move back and forth to and from the cloud), and the impact of a higher level of stress.4. Stability Testing – To assess the long-term reliability of each CSP5. Scalability Testing – To understand how well each CSP handles high object counts
  34. 34. Comparative Study Reports http://www.nasuni.com/downloads/resources/87/the_state_of_cloud_storage.pdf
  35. 35. Comparative Study Reports http://www.nasuni.com/downloads/resources/87/the_state_of_cloud_storage.pdf
  36. 36. Comparative Study Reports http://www.nasuni.com/downloads/resources/87/the_state_of_cloud_storage.pdf

×