AWS Summit 2013 | Singapore - Understanding AWS Storage Options

  • 449 views
Uploaded on

AWS provides multiple storage options to meet your varying needs. This presentation provides an overview of how AWS Cloud storage services can be used to support application development and delivery, …

AWS provides multiple storage options to meet your varying needs. This presentation provides an overview of how AWS Cloud storage services can be used to support application development and delivery, backup, archive, disaster recovery, and virtualized compute.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
449
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Jeff Putt Business Development Manager, AWS APAC 18 July, 2013 Understanding AWS Storage Options
  • 2. CHOOSING THE RIGHT DATA STORAGE SOLUTION
  • 3. AWS Storage Options Object Storage Block Storage Connect AWS Storage with On-Premises Environments For NoSQL and Relational Database storage, catch the dedicated sessions + +
  • 4. We are sincerely eager to hear your feedback on this presentation and on re:Invent. Please fill out an evaluation form when you have a chance.We are constantly producing more data
  • 5. We are sincerely eager to hear your feedback on this presentation and on re:Invent. Please fill out an evaluation form when you have a chance. From all types of industries
  • 6. #1 Object Storage ●○○
  • 7. Amazon S3 Simple Storage Service
  • 8. 99.999999999% Durability
  • 9. Over 2 Trillion Unique Customer Objects
  • 10. 1.1 Million Peak Transactions Per Second
  • 11. S3 storage costs as low as 5½ cents per GB
  • 12. 1. High Durability 2. Unlimited Capacity 3. Low latency 4. High Volume Static Websites 5. Security & Compliance 6. Delivery via CDN S3 gives you So you can build Mobile Apps Music Apps Video Networks Video Apps Backup Apps Photo Apps Static Websites Ad Networks Anything else you’d like!
  • 13. Amazon Glacier Low-Cost Archiving Service
  • 14. 1¢ Per GB / Month
  • 15. $120 Per TB / Year
  • 16. 99.999999999% Durability
  • 17. 3−5 Hours Data Retrieval
  • 18. Storage Costs vs. Retrieval Costs
  • 19. 1. High Durability 2. Low cost 3. 3-5 hour latency Glacier gives you So you can Archive files Build Hierarchical Storage Management in the cloud
  • 20. Amazon S3 and Glacier can help you build… GamingCommerce Digital Media Distribution Websites Social Media Photo/Video Sharing and Backup Big Data
  • 21. Store unlimited objects in Amazon S3 and Amazon Glacier S3 Glacier
  • 22. Access both via SDKs and via third-party software S3 Glacier Internet Web Services API HTTP(S) Via SDK Via 3rd parties
  • 23. Or send S3 files directly to Glacier S3 Glacier
  • 24. Serve static content at high volumes
  • 25. Deliver content with lower latency via Amazon CloudFront
  • 26. Samsung SmartHub delivers application content for TVs, Blu-Ray players, tablets, and phones
  • 27. Adds over 20,000 tracks of music a day
  • 28. Shazam hit 500,000 writes per second during the Super Bowl
  • 29. Nasdaq FinQloud is a WORM compliant archive for Broker Dealer financial data
  • 30. Spuul <3 AWS Michael Smith Jr CPO at Spuul.com July 18, 2013 @dreampipe
  • 31. What is Spuul? spool /spo͞ol/ Noun A cylindrical device on which something can be wound; a reel: "spools of electrical cable". Verb Wind (magnetic tape or thread) on to a spool: "he was trying to spool his tapes". Synonyms noun. reel - bobbin - coil verb. reel - wind
  • 32. Spuul is…
  • 33. Spuul Tech Challenges • Been operational for 1 year and 3 months – Went from zero to being a top Indian video player with large user bases in India, Pakistan, UK, USA, Australia, UAE, Singapore, Saudi Arabia, Canada & Malaysia • Lots of movie files – S3 seemed like a natural global file system • Need to store and deliver content – both web assets and mobile streaming • S3 stats – growing daily – 14 TB of original movie files – 2 TB of encoded mezzanine files – 35 GB of temp for QA work – 1 GB of screen grabs – 4 TB of mobile video segments and playlists
  • 34. CloudFront HD sent from India by plane QA remove Black Borders, Aspect Ratio Source File sent to S3 Launch Zencoder Process QA Settings, reduce Bitrate, remove Interlacing Spuul Backend link to Movie Assets Store Mezzanine in S3 Mezzanine File sent to Brightcove Zencoder use Mezzanine Spuul Processing Flow
  • 35. Spuul Tech • We use Zencoder for all encoding which is built on AWS and uses S3 for files • Everyone needs a DB and RDS works well for us • We capture all pertinent user and network activity with Dynamo DB, dump to S3 and store it – we figure over time it grows in value • We use Elastic Map Reduce to crunch our exported Dynamo data for daily/monthly dashboards, research and for some secret sauce • We need production environments for all of our interfaces/APIs but we also need multiple internal test environments and external test environments
  • 36. Why AWS? • AWS APAC HQ is in Singapore – Fast uploads to the data center – we have big files. Plus mail in disk import as needed – We also have regular meetings with local team, plus out of town experts from time to time • AWS innovates and iterates – We can’t predict the future but AWS products are leading edge which gives us room to grow – We love the constant product updates and price reductions • AWS helps us experiment – Every decision at a startup is to prove or disprove a hypothesis – with AWS global reach we can easily try things around the globe to quickly get to our proof points – good or bad • AWS is transparent – We try not to guess so with AWS we instrument everything – the data is our answer – Spuul is small – AWS allows us to automate just about everything – less bodies – more code
  • 37. Spuul <3 AWS
  • 38. #2 Block Storage ●●○
  • 39. Amazon EBS Elastic Block Storage
  • 40. Ephemeral storage is on every EC2 instance
  • 41. EBS volumes attach to EC2 instances
  • 42. EBS volumes persist and can be attached to other instances
  • 43. Multiple drives on one Amazon EC2 Instance
  • 44. 10GB 1TB 
  • 45. IOPS Provisioned
  • 46. Workloads on EBS Provisioned IOPS P-IOPS is designed to run transactional applications that require high and consistent IO such as: • Relational databases • NoSQL databases (e.g. MongoDB) • High performance file systems • Productivity applications (e.g. Microsoft Exchange) • Enterprise applications (e.g. CRM, ERP)
  • 47. EBS Snapshots
  • 48. Take snapshots of EBS volumes and keep them in S3
  • 49. Use Amazon EBS when you need • File system for an instance NTFS, ExtFS, RAID, LVM • Long-term persistent storage • Ability to change data frequently • Access to raw, unformatted block-level storage
  • 50. #3 Connect AWS Storage with On-Premises Environments ●●●
  • 51. Access both via SDKs and via third-party software S3 Glacier Internet Web Services API HTTP(S) Via SDK Via 3rd parties
  • 52. Common Data Storage Challenges and the AWS Opportunity Next Generation Enterprise Storage Internet Web Services API HTTP(S) AWS Direct Connect Block File Customer Data Center Storage Use Cases Archive Backup Disaster Recovery S3 Glacier
  • 53. Primary Block Storage Primary File Storage Archival Storage Disk-Based Backup Storage Tape Infrastructure & Management Replicated Storage for Disaster Recovery Offsite Locations Geo- Resilience Next Generation: Gateway & Cloud Storage Reduce SAN Footprint Reduce NAS Footprint Eliminate Architecture Hardware & Software Eliminate Backup Hardware & Software Eliminate Tape Recover in Cloud or any Data Center Eliminate Secondary Data Centers Reduce Complexity. Reduce Cost. Common Data Storage Challenges and the AWS Opportunity Next Generation Enterprise Storage – Benefits
  • 54. Common Data Storage Challenges and the AWS Opportunity S3 & Glacier – Designed to work with other AWS Services
  • 55. Storage Gateway – Connect On-Prem with the AWS Cloud 1. Local, low-latency access to the most frequently used files while storing all data in Amazon S3 (Cached-Volumes) Or 2. Scheduled off-site backups to Amazon S3 for on-premises data (Stored-Volumes)
  • 56. Use AWS Storage Gateway When You Need To • Backup your data • Synchronize data • Export data for migration • Set up a departmental fileshare • Conduct simple disaster recovery
  • 57. AWS Storage Options Object Storage Block Storage Connect AWS Storage with On-Premises Environments + +
  • 58. Performance-Oriented Cost-Oriented IT’S ALL ABOUT CHOICE
  • 59. Related Content Web resources: • http://aws.amazon.com/s3/ • http://aws.amazon.com/ebs/ • http://aws.amazon.com/glacier/ • http://aws.amazon.com/digital-media/
  • 60. Business Transformation Track