Data & Analytics - Session 4 - Understanding Storage Options


Published on

AWS provides multiple storage options to meet your varying needs. This presentation provides an overview of how AWS Cloud storage services can be used to support application development and delivery, backup, archive, disaster recovery, and virtualized compute.

James Brown, Business Development Manager, Storage, AWS
Stuart Wright, IT Director and Francis Hart, Systems Administrator, SEGA

Published in: Technology, Business
  • Be the first to comment

Data & Analytics - Session 4 - Understanding Storage Options

  1. 1. James Brown, Business Development ManagerChoosing the right datastorage options with AWS
  2. 2. Object StorageBlock StorageConnecting AWS Storage to On-premiseWhat are we going to cover?
  3. 3. We are sincerely eager to hearyour feedback on thispresentation and on re:Invent.Please fill out an evaluation formwhen you have a chance.We are constantly producing more data
  4. 4. We are sincerely eager to hearyour feedback on thispresentation and on re:Invent.Please fill out an evaluation formwhen you have a chance.From all types of industries
  5. 5. #1Object Storage●○○
  6. 6. Amazon S3Simple Storage Service
  7. 7. 99.999999999%Durability
  8. 8. Over 2 TrillionUnique Customer Objects
  9. 9. Over 1.1 millionPeak Transactions Per Second
  10. 10. Spotify adds over20,000 tracks aday.“Amazon S3 givesus confidence inour ability toexpand storagequickly while alsoproviding highdata durability.”Emil Fredriksson,operations director
  11. 11. Amazon S3Store objects from 1 byte to 5 terabytesUses standards based REST and SOAP protocolsHighly durable, fully managedDesigned to makeweb-scale computingeasierAmazon S3 - Unlimited Storage
  12. 12. Amazon S3 WebsitesEasily handle peak loads at low costPay as you go pricingInexpensive andHighly Scalable webhostingStatic Website Hosting
  13. 13. Real Clear Politicsserved up to 20xnormal demandwith S3 Websites
  14. 14. CloudFrontPay as you go pricing with no long termcommitmentsCaches content closer to your users tolower latency and better performanceIntegrated low latencycontent distributionnetworkDeliver content with lower latency with CloudFront
  15. 15. Storage costs as low as 5½ cents per GB
  16. 16. USE AMAZON S3WHEN YOU NEED:High DurabilityUnlimited Storage CapacityHigh ScaleHigh Volume Static WebsitesDelivery via CloudFront
  17. 17. Amazon GlacierLow-Cost Archiving Service
  18. 18. 1¢Per GB / Month
  19. 19. $125Per TB / Year
  20. 20. 99.999999999%Durability
  21. 21. 3-5 HoursData Retrieval
  22. 22. Storage CostsvsRetrieval Costs
  23. 23. Lifecycle PoliciesCreate rules on object prefixCreate rules to also delete expired objectAutomatically tier S3data in GlacierUsing Lifecycle policies to move data
  24. 24. Riverbed WhitewaterEliminates the burden of tapeImprove DRAdditional layer of securityReduce storage requirements by 10 to 30xA physical device canmanage integrationwith Glacier3rd Party Solution integration with Glacier
  25. 25. USE AMAZON GLACIERWHEN YOU NEED:Long term storage of infrequently accessed dataReduce costs of existing storageHigh durability
  26. 26. From theory intopractice withStuart Wright –Director ITFrancis Hart –Systems Architect
  27. 27. SEGAAWS Summit 2013Stuart Wright - Director IT & NetworksFrancis Hart – Systems Architect
  28. 28. • SEGA overview• Publishing & Development• Games & Data• Infrastructure past and present• What we use• Infrastructure close up• S3• Glacier• CloudFrontAgenda
  29. 29. SEGA overview• Who – Games development and publishing• What - Games – Platforms• Where - Publishing headquarters
  30. 30. Publishing & Development• Game publishing, Game hosting• QA, Logistics, Marketing, Sales, PR, etc• Game development• SEGA EuropeFM , Total War, Company of Heroes, Sonic mobile
  31. 31. Games & Data• Changing games• Online components• Analytics• Game evolution• Live systems
  32. 32. Infrastructure Past and Present• SEGA Datacentres• Physical Hardware• Flexibility and reliability• Contracts• Cloud IAAS
  33. 33. What we use• AWS storage options• Application development and delivery• Backup, Archive, Disaster recovery
  34. 34. Infrastructure close up• Shift to Virtualization and Cloud Services• Consumable computing resource• Consumption and production of data• Retention of data
  35. 35. SEGA S3• Shared Storage for EC2• Static content• Configuration• Interface and Integration with otherstorage options
  36. 36. SEGA S3• Example:EC2 Cloud Formation Cluster
  37. 37. SEGA Glacier• Backups• Data Retention• Legal Retention• RAW Analytic Data Storage
  38. 38. SEGA Glacier• Example:Log Retention (S3  Glacier)
  39. 39. SEGA CloudFront• Simple• Cost Effective• Reliable
  40. 40. SEGA CloudFront• Static Web Content• Game Patching• Media Content
  41. 41. #2Block Storage○●○
  42. 42. Amazon EBSElastic Block Store
  43. 43. Ephemeral storage is on every EC2 instance
  44. 44. Amazon EBSParticularly suited for applications that requirea database, file system, or access to raw blocklevel storage.Can be attached to a running instanceNetwork attachedpersistent block levelstorage volumesEBS Drives can be moved between instances
  45. 45. How do our customers use EBS?EnterprisesEnterpriseworkloads are builton block storageOracle, SAP,MicrosoftGamingcustomersVery highperformancedatabases.500,000 IOPS for anew game launchSocial Network/ Mobile AppsVery highperformance andconsistent I/O forNoSQL andrelational DBsMarketing /AnalyticsHigh performanceI/O databases
  46. 46. Multiple drives on one Amazon EC2 instance
  47. 47. 10GB  1TB
  48. 48. Provisioned IOPSAmazon EBS
  49. 49. Provisioned IOPSP-IOPS is designed to run transactionalapplications that require high andconsistent I/OWorkloads on Provisioned IOPsUse CasesRelational DatabasesNoSQL Databases (eg MongoDB)High Performance File SystemsEnterprise Workloads (eg CRM, MS Exchange, ERP)
  50. 50. Provisioned IOPS- For steady I/O use Provisioned IOPS- For bursty I/O, run the numbersOptimized for price/performance
  51. 51. EBS SnapshotsAmazon EBS
  52. 52. EBS Snapshots- Snapshots from EBS are stored in Amazon S3- Rollback to a previous version- Can be copied between regionsEBS Snapshots – Stored in Amazon S3
  53. 53. USE AMAZON EBSWHEN YOU NEED:Filesystem for an instanceNTFS, ExtFS, RAID, LVM…Long-term persistent storageData changes frequentlyAccess to raw, unformattedblock-level storage
  54. 54. #3Connecting AWS Storage toOn-Premise Environments○○●
  55. 55. AWS Storage GatewayConnecting with on-premise
  56. 56. AWS Storage Gateway- Low latency access to frequently accessed data- Minimise the need to scale out on-premisestorage- Snapshots allow access to old dataUp to 150TBs supportedfrom a single StorageGatewayExtend your on-premise storage
  57. 57. USE AWS STORAGE GATEWAYWHEN YOU NEED:Synchronize data for disasterrecoveryDepartmental fileshareBackup your data
  58. 58. Oracle Secure Backup Module
  59. 59. Oracle Secure Backup (OSB) CloudModule allows customers to backupOracle Databases directly to Amazon S3using the Oracle Recovery Manager(RMAN)Backup Oracle into Amazon S3
  60. 60. Restore timesreduced from 15to 2½ hoursTook less that 1hour to configureper databaseOffered significantimprovements indurability ofbackups