Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AWS Summit Seoul 2015 - EBS 성능 향상 및 EC2 비용 최적화 기법

3,426 views

Published on

2015년 4월 21일 서울 코엑스에서 열렸던 AWS Summit Seoul 2015의 첫번째 트랙에서 양승도 아마존 웹서비스 솔루션스 아키텍트가 발표한 자료입니다.

Published in: Technology
  • Dating for everyone is here: ❤❤❤ http://bit.ly/2ZDZFYj ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ❶❶❶ http://bit.ly/2ZDZFYj ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

AWS Summit Seoul 2015 - EBS 성능 향상 및 EC2 비용 최적화 기법

  1. 1. SEOUL ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
  2. 2. ©2015,  Amazon  Web  Services,  Inc.  or  its  affiliates.  All  rights  reserved Amazon  Elastic  Block  Store  Deep  Dive &  AWS  Cost  Optimization 양승도 솔루션즈 아키텍트, 아마존 웹서비스 코리아
  3. 3. ©2015,  Amazon  Web  Services,  Inc.  or  its  affiliates.  All  rights  reserved Amazon  EBS (Elastic  Block  Store)
  4. 4. A  “normal”  hard  drive
  5. 5. EBS =
  6. 6. What  is  EBS? • Network  block  storage   • Designed  for  five  nines  of  availability • Attaches  to  Amazon  EC2  within  the  same   Availability  Zone • Provides  point-­in-­time  snapshots  to   Amazon  S3  
  7. 7. More  about  EBS • It’s  a  service! • It’s  independent  of  EC2 • It  has  regional  and  AZ  availability  goals – All  EBS  volumes  are  designed  for  99.999%  availability • Over  1  million  volumes  are  created  per  day   (average) • Over  3.3  trillion  I/Os per  day
  8. 8. 1TB 16  TBÚ March  2015
  9. 9. EBS General  Purpose  (SSD) Up  to  16  TB 10,000  IOPS Up  to  160  MB/s Provisioned  IOPS  (SSD) Up  to  16  TB 20,000  IOPS Up  to  320  MB/s
  10. 10. A  few  definitions… • IOPS:  Input/output  operations  per  second  (#) • Throughput:  Read/write  rate  to  storage  (MiB/s) • Latency:  Delay  between  request  and  completion  (ms) • Capacity:  Volume  of  data  that  can  be  stored  (GiB) • Block  size:  Size  of  each  I/O  (KiB)
  11. 11. EBS  volume  types • General  Purpose  (SSD) • Provisioned  IOPS  (SSD) • Magnetic   When  performance  matters,  use  SSD-­backed  volumes
  12. 12. EBS  SSD  volumes • Applies  to  both  General  Purpose  and   Provisioned  IOPS • IOPS  measured  up  to  256  KiB • Single-­digit  ms latency • Designed  for  99.999%  availability
  13. 13. EBS  General  Purpose  volumes  (SSD) New  default  volume  type  for  EBS Every  volume  can  burst  up  to  3,000  IOPS • Larger  volumes  can  burst  for  longer  periods 3  IOPS  per  GB  baseline  performance,   maximum  of  10,000  IOPS 99%  performance  consistency Up  to  160  MB/s  throughput
  14. 14. (2)  Max  I/O  credit  per  bucket  is  5.4M (1)  Always  accumulating  3   IOPS  per  GB  per  second   (3)  You  can  spend  up  to   3,000  IOPS  per  second   Understanding  General  Purpose  (SSD)  bursting Baseline  performance  =  3  IOPS  per  GB
  15. 15. General  Purpose  (SSD)  volumes  example Microsoft  Windows  30  GB  boot  volume:   • Gets  initial  I/O  credit  of  5.4M   • Could  burst  for  up  to  30  mins @  3,000  IOPS • Always  accumulating  90  I/O  credits  per   second
  16. 16. m3.medium  US-­EAST1 Volume type Boot time Access time OS GP2 3:31 4:33 Windows Server   2012 Magnetic 4:30 7:16 Windows Server   2012 GP2 0:36 0:45 CentOS6 Magnetic 0:57 1:16 CentOS6 40%  Reduction  in  boot  times  by  using  General  Purpose  SSD Instance  Boot  Time
  17. 17. Always  use  General  Purpose  (SSD)  for  boot  volumes
  18. 18. 1  TB  PIOPS  volume  with  4,000  IOPS  =  $526.40  per  month  per  volume GP2  1  TB  volume  with  3,000  IOPS  =  $102.40 GP2  2  x  500  GB  volume  at  3,000,  Burst  to  6,000  = $102.40 80%  cost  savings,  50%  more  peak  I/O  with   General  Purpose  SSD Database  volume
  19. 19. EBS  PIOPS  (SSD)  volumes • Best  for  I/O  intensive  databases  that  require  highest   consistency • Throughput  up  to  320  MB/s • Provision  up  to  20,000  IOPS  per  volume   (supports  IOPS:GB  ratio  of  30) • Designed  for  99.9%  performance  consistency
  20. 20. EBS  Magnetic  volumes • Best  for  cold  workloads  (rarely  accessed  data  that  needs always-­on  access) • IOPS:  ~100  IOPS  steady-­state,  with  best-­effort  bursts • Throughput:  variable  by  workload,  best  effort  to  10s  of  MB/s • Latency:  Varies,  reads  typically  ~20-­40  ms,  writes  typically   ~2-­10  ms
  21. 21. Price Performance EBS Magnetic General  Purpose Provisioned  IOPS Use  cases Infrequent data  access Boot  volumes Small  to  med  DBs Dev  and  Test I/O  intensive Relational  DBs NoSQL  DBs Storage  media Magnetic  disk-­backed SSD-­backed SSD-­backed Max IOPS 40–200  IOPS 10,000  IOPS 20,000  IOPS Latency  (random   read) 20 ~  40  ms 1 ~  2  ms 1 ~  2  ms Availability Designed  for  99.999% Designed  for  99.999% Designed  for  99.999% Price $.05/GB-­month $.05/million  I/O $.10/GB-­month $.125/GB-­month $.065/provisioned  IOPS
  22. 22. Performance
  23. 23. Performance  optimization  is  measured  by: IOPS:  Read/write  I/O  rate  (IOPS) Latency:  Time  between  I/O  submission   and  completion  (ms) Throughput:  Read/write  transfer  rate   (MB/s);;  throughput  =  IOPS  X  I/O  size
  24. 24. Four  key  components  of  performance  optimization   1.  EC2  instance 2.  I/O 4. EBS 3.  Network  link
  25. 25. Tools  available  for  performance  tuning: 1. EC2  instance:  Network  bandwidth  (Mbps) 2. EBS-­optimized  instance:  EC2  instance  option  (On/Off) 3. Workload:  Block  size,  read/write  ratio,  serialization 4. Queue  depth:  The  number  of  outstanding  I/Os 5. RAID:  Stripe  volumes  to  maximize  performance 6. Pre-­warming:  Eliminate  first-­touch  penalty
  26. 26. Compute-­optimized  – C3/C4   Memory-­optimized  – R3 General-­Purpose  – M3 EBS EC2 Select  the  EC2  instance  that  has  the  right  Network,   RAM,  and  CPU  resources  for  your  applications 1.  EC2  instance
  27. 27. Most  instance  families  supports  the  EBS-­optimized  flag EBS-­optimized  instances  now  support  up  to  4  Gbps • c4.8xlarge,  d2.8xlarge • Drive  32,000  16K  IOPS  or  500  MB/s   Other  *.8xlarge  instances  support  10  Gbps network • Max  IOPS  per  node  supported  is  ~48,000  IOPS  @  16K 2.  EBS-­optimized  instance
  28. 28. Use  EBS-­optimized  instances  for   consistent  EBS  performance
  29. 29. I/O  size:   • 4  KB  to  64  MB I/O  pattern: • Sequential  and  random   I/O  type: • Read  and  write     I/O  concurrency:   • Number  of  concurrent  I/O EBS  SSD-­backed  volumes  measure  I/O  size  up  to  256  KiB EBS  SSD-­backed  volumes  deliver  same  performance  for  read  and  write 3.  Workload
  30. 30. EBS  IOPS  and  throughput  limits 20,000  IOPS   PIOPS  volume   20,000 IOPS 320  MB/s   throughput You  can  achieve  20,000  IOPS  when   driving  smaller  I/O  operations You  can  achieve  up  to  320  MB/s   when  driving  larger  I/O  operations
  31. 31. EBS  IOPS  and  throughput  limits 8,000  IOPS   PIOPS  volume   8,000  IOPS 320  MB/s   throughput 8,000  x  64  KB=512  MB/s 5,000  x  64  KB  =  320  MB/s 1,250  x  256  KB  =  320  MB/s 8,000  X  8  KB  =  64  MB/s 8,000  X  16  KB  =  128  MB/s 16,000  x  8  KB  =  128  MB/s 8,000  x  32  KB  =  256  MB/s
  32. 32. Block  (I/O)  size  determines  whether  your   application  is  IOPS  bound  or  throughput  bound    
  33. 33. 4.  Queue  depth An  I/O  operation EBS After  it’s  gone,  it’s  gone EC2 Queue  depth  is  the  pending  I/O  for  a  volume
  34. 34. Important  Amazon   CloudWatch metrics:   • IOPS  and  bandwidth • Latency   • Queue  depth Monitoring  EBS  volumes
  35. 35. 0.075 35.1 2.09 1,865   4,152   3,851   -­ 500   1,000   1,500   2,000   2,500   3,000   3,500   4,000   4,500   0 5 10 15 20 25 30 35 1 4 8 12 16 20 24 28 32 Latency    TP90  (ms) Queue  depth 16  KB  random  read  IOPS,  latency  across  various  queue  depths Latency  (TP90) Avg  Read  IOPS IOPS Queue  depth  between  4  and  8  has  the  optimal  IOPS  and  latency  performance Queue  depth  vs.  Random  read  latency
  36. 36. Queue  depth  vs.  Random  write  latency 0.08 7.71 845 4,152 0 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 0 1 2 3 4 5 6 7 8 9 10 1 4 8 12 16 20 24 28 32 Latency    TP90  (ms) Queue  depth 16  KB  random  write  IOPS,  latency  across  various  queue  depths Latency  (TP90) AvgIOPS IOPS Write  latency  queue  depth  and  IOPS  interaction  is  similar  to  that  of  read  latency
  37. 37. Optimal  queue  depth  to  achieve  lower  latency  and  highest  IOPS   is  typically  between  4-­8;;  ~1  queue  depth  per  500  IOPS EBS-­optimized  instances  provide  consistent  latency  experience Use  SSD  volumes  with  latest-­generation  EC2  instances
  38. 38. 5.  RAID Increases  performance,  or  capacity,  or  both Over  320  MB/s  or  20,000  IOPS,  striping  needed Don’t  mix  volume  types Typically  RAID  0  or  LVM  stripe Avoid  RAID  for  redundancyEBS EC2
  39. 39. • Eliminates  first-­access  penalty • Typically  5%,  extreme  worst  case  of  50%  performance  reduction  in  IOPS  and  latency   when  volumes  are  used  without  pre-­warming:   – Performance  is  as  provisioned  when  all  the  chunks  are  accessed   • Recommendations  before  benchmarking: – For  new  volumes:   • Linux:  DD  write • Windows:  NTFS  full  format – Takes  roughly  an  hour  to  pre-­warm  1  TB  PIOPS/General  Purpose  (SSD)  volumes • Always  check  latest  documentation   http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-­prewarm.html 6.  Pre-­warming
  40. 40. Use  large  block  size  to  speed  up  your  pre-­warming Example:  sudo dd if=/dev/zero   of=/dev/xvdf conv=notrunc bs=1M
  41. 41. Workload/   software Typical  block   size Random/ Seq? Max  EBS  @  500   MB/s  instances Max  EBS  @   1  GB/s  instances Max EBS  @  10  GB/s   instances Oracle  DB Configurable:2  KB –16  KB Default  8  KB random ~7,800  IOPS ~15,600  IOPS ~96,000  IOPS Microsoft  SQL   Server 8  KB  w/  64  KB   extents random ~7,800  IOPS ~15,600  IOPS ~80,000  IOPS MySQL   16  KB random ~4,000  IOPS ~7,800  IOPS ~48,000  IOPS PostgreSQL 8  KB random ~7,800  IOPS ~15,600  IOPS ~96,000  IOPS MongoDB 4  KB serialized ~15,600  IOPS ~31,000  IOPS ~96,000  IOPS Apache   Cassandra 4  KB random ~15,600  IOPS ~31,000  IOPS ~96,000  IOPS GlusterFS 128  KB sequential ~500  IOPS ~1,000  IOPS ~6,000  IOPS Cheat  sheet  sample:  Storage  workloads  on  AWS
  42. 42. EBS-­optimized  instance Four  key  components:  balanced    (Oh,  YEAH!!) EC2 A  “boatload”  of  I/O Right-­sized  EBS
  43. 43. ©2015,  Amazon  Web  Services,  Inc.  or  its  affiliates.  All  rights  reserved AWS  Cost  Optimization
  44. 44. How  to  make  a  lower  AWS  bill?
  45. 45. Strategy  1:  Do  nothing
  46. 46. AWS  pricing  philosophy Ecosystem Global Footprint New Features New Services More AWS Usage More Infrastructure Lower Infrastructure Costs Reduced Prices More CustomersInfrastructure Innovation 48  price   reductions   since  2006Economies of Scale
  47. 47. Strategy  2:  Do  almost  nothing
  48. 48. AWS  Trusted  Advisor aws.amazon.com/premiumsupport/trustedadvisor/ Free  with  Business  or  Enterprise  Support
  49. 49. Optimization Cost  reduction
  50. 50. Strategy  3:  Optimize  Architecture (Architecting  for  low  cost)
  51. 51. 1:  Turn  off  unused  instances
  52. 52. 2:  Use  Auto  Scaling
  53. 53. Poll:  Who  uses  Auto  Scaling?
  54. 54. 3:  Use  Reserved  Instances
  55. 55. 4:  Use  Spot  Instances
  56. 56. Poll:  Who  uses  Spot  Instances?
  57. 57. Spot  Instance  example On-­Demand: $0.24 $0.028    (11.7%) $0.026    (10.8%) $3.28 (1367%)
  58. 58. • Reduced  redundancy  storage  class – 99.99%  durability  vs.  99.999999999% – Up  to  20% savings – Everything  that  is  easy  to  reproduce – Use  Amazon  SNS  lost  object  notifications • Amazon  Glacier  storage  class – Same  99.999999999%  durability – 3  to  5  hours  restore  time – Up  to  64% savings – Archiving,  long-­term  backups, and old  data • Use  life-­cycle  rules 5:  Use  Amazon  S3  storage  classes
  59. 59. • Read/write  capacity  units (CUs)  determine most  of  DynamoDB  cost • By  optimizing  CUs,  you  can  save  a  lot  of  money • But: – Need  to  provision  enough  capacity  to  not  run  into  capacity  errors – Need  to  prepare  for  peaks – Need  to  constantly  monitor/adjust 6.  Optimize  Amazon  DynamoDB capacity  units
  60. 60. Dynamic  DynamoDB
  61. 61. DynamoDB  optimization  example Caching/Optimization: 80%  saved Cache flush Dynamic DynamoDB: 20%  saved Growth  + new  features
  62. 62. EC2 1. 2. 3.4. Buffering  DynamoDB with  SQS
  63. 63. 7.  Offload  your  architecture • The  more you  can  offload,  the  less infrastructure  you  need  to  maintain,  scale, and  pay for • Three  easy  ways  to  offload: – Use  Amazon  CloudFront – Introduce  caching – Leverage  existing  Amazon  web  services
  64. 64. Offload  popular  traffic  to  Amazon  S3,  CloudFront
  65. 65. Leverage  existing  services • Amazon  RDS,  Amazon  DynamoDB  or  Amazon   ElastiCache  for  Redis,  Amazon  Redshift – Instead  of  running  your  own  database • Amazon  CloudSearch – Instead  of  running  your  own  search  engine • Amazon  Elastic  Transcoder • Amazon  Elastic  MapReduce • Amazon  Cognito,  Amazon  SQS,  Amazon  SNS, Amazon  Simple  Workflow  Service,  Amazon  SES, Amazon  Kinesis,  and  more  … Simple,  more  reliable,  lower  cost
  66. 66. Cost  monitoring  and  analysis
  67. 67. AWS  Billing  Console
  68. 68. AWS  Cost  Explorer
  69. 69. AWS  billing  alerts
  70. 70. Let’s  recap 1. Turn  off unused  instances 2. Use  Auto  Scaling 3. Use  Reserved  Instances 4. Use  Spot  Instances 5. Leverage  Amazon  S3  storage  classes 6. Optimize Amazon  DynamoDB  capacity  units 7. Offload your  architecture
  71. 71. …  and  remember  to  iterate!
  72. 72. SEOUL

×