Using AWS CloudFront with S3 at SMARTSTUDY

2,469 views

Published on

AWS 한국 사용자 그룹 2013년 1회 세미나에서 발표한 자료입니다.

Published in: Technology

Using AWS CloudFront with S3 at SMARTSTUDY

  1. 1. Using CloudFront with S3 at SMARTSTUDY Amazon Web Service Korean User Group Park Hyun-woo, a SCV of SMARTSTUDY / ez@smartstudy.co.kr
  2. 2. What is S3 and CF?● S3 : Simple Storage Service ○ Extreme durability. ○ Multiple copies in selected region. ■ vs EBS : Multiple copies in single available zone.● CF : CloudFront ○ Content Delivery Network(CDN) service. ○ Distribute from origin to AWS Edge Network ■ http://aws.amazon.com/cloudfront/#details ■ About 30 edges in global. (only in Earth, lol)
  3. 3. Why choose these?● We serve large multimedia contents to global market in mobile application era.● We need redundant, fast and reasonable price content delivery service aims globally.● We have not enough human resource to maintain various system.
  4. 4. Flash Movie
  5. 5. TranscoderFlash Gnash FFmpeg Package ( encrypted )Movie qtfaststart Thumbnails npk Video S3 Bucket s3cmd ( tokyo )
  6. 6. npk Video S3 Bucket s3cmd ( tokyo ) {Edge Edge Edge CloudFront Edge Edge
  7. 7. How it works?● Copy to S3 ○ ~100 files, just use Cyberduck or other GUI tool. ○ Or, you have to use s3fs or s3cmd. ○ If you need sync or invalidation, use s3cmd. ○ We have ~30,000 files. Thus, s3cmd. ○ We chose Tokyo bucket for faster uploading.● s3cmd ○ Awesome! ○ Works on S3 and CloudFront.
  8. 8. How it works? (cont.)● s3cmd ○ Can do almost everything as like as AWS console. ○ Support INI-style configuration file. ○ And, magical sync command. (yeah!)● s3cmd sync ○ Support glob-style exclude / include option. ○ ... rexclude / rinclude for regular expression. ○ Support automatic invalidation request. ○ ... with rsync-like dry-run option. (-n, --dry-run)
  9. 9. How it works? (cont.)● s3cmd sync ○ Basically, sync based on MD5 information. ○ (!) Over 15MB files, using only filesize. ○ Our 30,000 files, 60GB = 8 min to dry-run. ( env : i7 2500K + SATA HDD to bucket in tokyo )● s3cmd sync --cf-invalidate ○ Call invalidation API automatically. ○ (!) Simultaneous invalidation is up to 3. ○ (!) Free for ~1,000 files per month. ○ Check it up by s3cmd cfinvalinfo
  10. 10. So, is it really? FAST{ REDUNDANT NOT EXPENSIVE
  11. 11. Its fast enough.8.0MB /s
  12. 12. Its very redundant.BETTER THAN YOUR OWNSERVER, EVER
  13. 13. And NOTEXPENSIVE
  14. 14. Bill for Bill forDomestic CDN CloudFrontBased on Based onPeak traffic Total traffic● 350Mbps/@peak ● 10TB/month● $6.00/Mbps ● $0.209/GB = $2,100 = $2,090* * without transfer fee for deploying origin bucket to edges
  15. 15. STILLEXPENSIVE?
  16. 16. * Photo from Linkedin page of Lee, Jung-in / Amazon Korea
  17. 17. For over 10TB, RESERVEDCapacity Pricing
  18. 18. Tips● Smaller ○ Encode media with optimal options. ○ Use dedicated option for each platform. ○ Storage is cheaper than transfer fee!● Static ○ Do not deploy volatile objects to CloudFront. ○ Invalidation needs ~10min, often. ○ Were still using Nginx for them.● Streamable ○ Mobile networks are still slow. ○ Do not make users to wait until finishing download.
  19. 19. Link● Gnash ○ http://www.gnu.org/software/gnash/● FFmpeg ○ http://www.ffmpeg.org/● qtfaststart ○ https://github.com/danielgtaylor/qtfaststart● npk ○ https://github.com/lqez/npk● s3cmd ○ http://s3tools.org/
  20. 20. Using CloudFront with S3 at SMARTSTUDY Amazon Web Service Korean User Group Park Hyun-woo, a SCV of SMARTSTUDY / ez@smartstudy.co.kr

×