Highly Available Web Properties in Aws

2,710 views
2,710 views

Published on

Published in: Technology, Business
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,710
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
185
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide
  • Elasticity is a fundamental property of the CloudThe difference between a classic architecture and a cloud architecture often resides in elasticity implementation
  • Ideally, you would need a cursor to dynamically adjust to the desired capacity level.It's exactly what EBK do … automatically.
  • Pour illustrer le launch early
  • 00:47:00
  • Is it open sourced yet ?
  • AUDITABILITY = No central location to track the status of a task
  • Montrerqu’apartir du moment oudoit faire des decisions, ilfaut considerer l’option SWFBubbles are steps, or activities. [workers]Arrows are control decisions, or coordination logic. [decider]
  • Activities, or steps
  • control decisions, or coordination logic
  • Writing a decider requires you to review the state of the workflow. The decider itself is stateless but SWF keeps the state and tells the decider about what has happened.[Point out that a decider can return several decisions in the same call. This allows for parallel processing.]To write workers and deciders you can use the SWF SDK (provided for Java, .NET, PHP) or call the API directly, but to make this easier [CUE NEXT SLIDE]
  • Compare this to what was required to write an SQS queue worker: Now the worker might have to send heartbeats; it also needs to take care of informing SWF if the work has been successful or not. This is required for SWF to keep track.
  • Create distinct Security Groups for each Amazon EC2 clusterUse group-based rules for controlling access between layersRestrict external access to specific IP rangesEncrypt data “at-rest”, encrypt data “in-transit” (SSL)Use Identity and Access Management (IAM)Rotate your AWS CredentialsUse Multi-Factor AuthenticationUse Amazon Virtual Private Cloud (VPC)
  • Filter by IP range, port range, security group, role
  • PCI DSS Level 1AWS satisfies the requirements under PCI DSS for shared hosting providers. AWS also has been successfully validated against standards applicable to a Level 1 service provider under PCI DSS Version 2.0. Merchants and other PCI service providers can use the AWS PCI-compliant technology infrastructure for storing, processing, and transmitting credit card information in the cloud, as long as those customers create PCI compliance for their part of the shared environment. Amazon Elastic Compute Cloud (EC2), Amazon Simple Storage Service (S3), Amazon Elastic Block Storage (EBS) and Amazon Virtual Private Cloud (VPC) were included as part of this validation. Under the same circumstances, other enterprises can also benefit by running their applications on other PCI-compliant technology infrastructure. AWS provides additional information and frequently asked questions about its PCI compliance on its web site.
  • cloud-hosted search service from AWSfully managed search service on the cloudscales automaticallyeliminates complex managementsupport structured or unstructured text documents. Why ?huge explosion on amount of data created everyday. structured or not. 
  • Amazon CloudSearch builds an index and picks the appropriate initial search instance type to ensure that your index can be stored in RAM. As your data volume grows, Amazon CloudSearch will scale your search domain to a larger search instance type (or partition your index across multiple instances if you are already on the largest search instance type).As with data volume, Amazon CloudSearch automatically scales your search domain to meet your traffic demands. When a search instance reaches over 80% CPU utilization, CloudSearch scales up your search domain by adding a search instance to handle the increased traffic. Conversely, when a search instance reaches below 30% CPU utilization, CloudSearch scales down your search domain by removing the additional search instances in order to minimize costs.
  • Pour illustrer le launch early
  • Highly Available Web Properties in Aws

    1. 1. Building HighlyAvailable, Scalable Web Properties with AWS Joe Ziegler │Technical Evangelist @jiyosub
    2. 2. 1. ELASTICITY2. DESIGN FOR FAILURE3. LOOSE COUPLING4. SECURITY5. PERFORMANCE
    3. 3. # 1ELASTICITY ●○○○○
    4. 4. AMAZON EC2ELASTIC COMPUTE CLOUD
    5. 5. AMAZON CLOUDWATCHMONITORING FOR AWS RESOURCES
    6. 6. AUTO SCALINGSCALE UP/DOWN EC2 CAPACITY
    7. 7. ELASTIC LOAD BALANCINGNETWORK TRAFFIC DISTRIBUTION
    8. 8. 6 am
    9. 9. 10 am
    10. 10. 10 am
    11. 11. 10 am
    12. 12. 7 pm
    13. 13. 7 pm
    14. 14. 7 pm
    15. 15. without elasticity you cant accelerate
    16. 16. what you really need is… …adjustable capacity
    17. 17. # 2DESIGN FOR FAILURE ●●○○○
    18. 18. « Everything fails all the time » Werner Vogels CTO of Amazon
    19. 19. YOUR GOALApplications should continue to function even if the underlying physical hardware fails or is removed or replaced
    20. 20. Avoid single points of failure.Assume everything fails, and designbackwards.
    21. 21. Avoid single points of failure.Assume everything fails, and designbackwards.
    22. 22. AMAZON RDSRELATIONAL DATABASE SERVICE
    23. 23. AMAZONROUTE 53DOMAIN NAME SERVICE
    24. 24. AMAZON RDSMULTI-AZDEPLOYMENT
    25. 25. AMAZON CLOUDWATCH ALARMS
    26. 26. AWS BUILDING BLOCKSInherently Fault-Tolerant Services Fault-Tolerant with the right architecture Amazon S3  Elastic Load Balancing  Amazon EC2 Amazon SimpleDB  AWS IAM  Amazon EBS Amazon DynamoDB  AWS Elastic  Amazon RDS Amazon CloudFront Beanstalk Amazon SWF  Amazon VPC  Amazon Amazon SQS ElastiCache Amazon SNS  Amazon EMR Amazon SES  Amazon CloudSearch Amazon Route53
    27. 27. NETFLIXCHAOS MONKEY
    28. 28. # 3 LOOSECOUPLING ●●●○○
    29. 29. BUILD LOOSELYCOUPLED SYSTEMS The looser the are coupled, the bigger they scale
    30. 30. Create independent components
    31. 31. Create independent componentsDesign everything as a Black Box
    32. 32. Create independent componentsDesign everything as a Black BoxThink in terms of services
    33. 33. TRANSCODERECEIVE & PUBLISH
    34. 34. TRANSCODERECEIVE & PUBLISH QUEUE
    35. 35. AMAZON SQSSIMPLE QUEUE SERVICE
    36. 36. START CHECK REJECT STOP VIDEO YES NO TOO SPAM PUBLISH SPAM? LONG? CHECK & NOTIFYYES NO SHORTEN TRANSCODE VIDEO
    37. 37. MAINTENANCEAUDITABILITYFLEXIBILITYLOW-LEVEL
    38. 38. WHAT ARE WETRYING TO DO?
    39. 39. START CHECK REJECT STOP VIDEO YES NO TOO SPAM PUBLISH SPAM? LONG? CHECK & NOTIFYYES NO SHORTEN TRANSCODE VIDEO
    40. 40. START CHECK REJECT STOP VIDEO YES NO TOO SPAM PUBLISH SPAM? LONG? CHECK & NOTIFYYES NO SHORTEN TRANSCODE VIDEO
    41. 41. START CHECK REJECT STOP VIDEO YES NO TOO SPAM PUBLISH SPAM? LONG? CHECK & NOTIFYYES NO SHORTEN TRANSCODE VIDEO
    42. 42. TASKSDECISIONS STATELESS ! HISTORY
    43. 43. STATELESS SCALES HORIZONTALLY
    44. 44. AMAZON SWF WORKFLOW SERVICE FORSCALABLE, RESILIENT APPLICATIONS
    45. 45. AMAZON SWF Keeps track of :  State  Executed tasks  Timeouts  Errors
    46. 46. WORKFLOWACTORS
    47. 47. DECIDERS COORDINATION LOGIC1. Poll for work on a decision list Long polling: 60 seconds2. Evaluate workflow execution history SWF sends full history in JSON format3. Return decision to Amazon SWF Usually scheduling another task
    48. 48. WORKERS COORDINATION LOGIC1. Poll for work on a specific task list Long polling: 60 seconds2. Execute works, send heartbeats SWF sends input data from deciders3. Return success / failure Detailed data can be provided to deciders
    49. 49. ALLHORIZONTAL SCALING PATTERNS APPLY
    50. 50. NO NEW LANGUAGE TO LEARNYOUR CODE IS YOUR WORKFLOW LANGUAGE SWF MAINTAINS STATE
    51. 51. AWS FLOW FRAMEWORKJava Library • Entire workflow can be expressed in sequential code • Integrated with Java Utils API
    52. 52. CHAINED TASKS WITHOUT DECISIONS?RECEIVE TRANSCODE NOTIFY use AMAZON SQS
    53. 53. TASK GRAPH WITH DECISIONS? GOOD OKRECEIVE CHECK SPAM TRANSCODE VIDEO LENGTH CHECK LONG SPAM SHORTEN PUBLISH REJECT VIDEO & NOTIFY use AMAZON SWF
    54. 54. # 4SECURITY ●●●●○
    55. 55. AWS IAMIDENTITY AND ACCESS MANAGEMENT
    56. 56. MULTI-MULTI-FACTORAUTHENTICATION
    57. 57. SECURITY GROUPS LAYERED SECURITY
    58. 58. CERTIFICATIONS& COMPLIANCE SAS 70 Type II audit ISO 27001 Certification Sarbanes-Oxley SOX HIPAA healthcare FISMA US Federal Government DIACAP MAC III Sensitive IATO
    59. 59.  PCI DSS Compliant LEVEL 1 SERVICE PROVIDER
    60. 60. # 5PERFORMANCE ●●●●●
    61. 61. VERTICALSCALING
    62. 62. CLUSTER COMPUTEEIGHT EXTRA LARGE 2x Intel Xeon E5-2670, eight-core Sandy Bridge architecture 60.5 GB of memory – 3.3 TB of storage
    63. 63. CLUSTER GPUQUADRUPLE EXTRA LARGE Intel Xeon X5570, quad-core 2x Nehalem architecture NVIDIA Tesla Fermi 2x M2050 GPUs 22 GB of memory – 1.7 TB of storage
    64. 64. AMAZONELASTICACHEMEMCACHED COMPATIBLE IN-MEMORY CACHE
    65. 65. SEARCHENGINES
    66. 66. Relevance & Ranking
    67. 67. Faceting
    68. 68. RangeSearching
    69. 69. AMAZONCLOUDSEARCHFULLY-MANAGED SEARCH SERVICE
    70. 70. +
    71. 71. DEMOCRATIZE SEARCH Amazon CloudSearch delivers a fully-managed search service in the cloudthat can be set up and running in less than 1 hour, with automatic scaling for data & traffic, at a price starting at less than $100 per month.
    72. 72. MEDIA SHARINGREFERENCE ARCHITECTURE
    73. 73. 1. ELASTICITY2. DESIGN FOR FAILURE3. LOOSE COUPLING4. SECURITY5. PERFORMANCE
    74. 74. « If youre not embarrassed when you ship your first version you waited too long » Matt Mullenweg CEO & Founder of WordPress
    75. 75. aws-apac-marketing@amazon.com
    76. 76. AWS ANZ Customer Appreciation DayIs Coming to Sydney November 13 th #AWSCADAU

    ×