Scalable Application Development on AWS


Published on

Presentation from Application Developers Days 3 conference (Moscow, May 2012) about best practices in scalable application development on AWS.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Scalable Application Development on AWS

  1. 1. Scalable ApplicationDevelopment on AWS Mikalai Alimenkou 11.05.2012
  2. 2. Background• Java Technical Lead/Scrum Master at Zoral Labs• 7+ years in software development• 5+ years of working by Agile methodologies• Expert in Agile engineering practices• Founder and coach at XP Injection• Organizer of Selenium Camp, JEEConf, XP Days Ukraine, IT Brunch conferences
  3. 3. NoFuture!
  4. 4. Cloud computing• Scale capacity on demand• Turn fixed costs into variable costs• Clean APIs and conceptual models• Always available• Cost-effective• Reduced time to market• Focus on product
  5. 5. • Interesting startup• Completely in Java• Customer wants to use AWS New project arrived
  6. 6. AWS structure• Elastic Compute Cloud • Simple Storage Service (S3) • SimpleDB (EC2) • Elastic Block Store (EBS) • DynamoDB• Elastic Map Reduce (EMR) • Import/Export • Relational Database• Auto Scaling • Storage Gateway Service (RDS)• Elastic Load Balancing • ElastiCacheCompute Storage Database• Route 53 • Simple Queue Service (SQS) • CloudWatch• Virtual Private Cloud • Simple Notification Service • Elastic Beanstalk (VPC) (SNS) • CloudFormation• Direct Connect • Simple Email Service (SES) • Identity and Access• CloudFront • Simple Workflow Service Management (IAM) (SWS) • CloudSearchNetworking App Services Deployment
  7. 7. Why Java?• Runs everywhere• Full technological stack• Lots of tools for scalability• Many frameworks for cloud development
  8. 8. EC2 quick introduction $.08-$2.4 per server hour $.05 - $.12 per GB data transfer
  9. 9. Simple boot model AMI Instant-store AMI • Stateless Instance • Cheaper • Better IO • Easier to scale Cloud EBS boot AMI • Stateful • Stop/start support • Quick migration • Faster loading
  10. 10. Different API access levelsAWS management console AWS console tools REST API clients Plugins to IDEs and browsers
  11. 11. Elasticfox – my superhero!
  12. 12. Everything on AWS • Infrastructure • Testing • Staging • Production • Monitoring
  13. 13. • Easy and reliable server configuration with AMI• No work for system administrator• Any number of servers available• No waits for hardware• More experimenting and testing• Easy security model Happiness…
  14. 14. Look ma, no hands!
  15. 15. What about money?
  16. 16. EC2 charging model
  17. 17. What I have for such money?TestingServer DBServer Production Server
  18. 18. Ways to save money• Good and responsible team • Reuse instances when possible rounding usage time• Release instances when not more needed• Monitor and control running instances• Use reserved instances for continuously running jobs
  19. 19. EC2 physical distribution
  20. 20. • Hourly payment is invisible• Instances are virtual• Same type of machine, but different performance• Instance not immediately available• Images creation require “magical” skills• No multicast messaging
  21. 21. Amazon cares aboutyou … and other people
  22. 22. Everything is unreliable• Retry policies everywhere• Timeouts for all operations• Automatic recovery
  23. 23. EBS quick introduction• Raw data volumes from 1GB to 1TB• Replicated automatically• Snapshots to S3• May be used with different instances
  24. 24. EBS architecture
  25. 25. EBS fault tolerance
  26. 26. EBS cost model• Not so cheap• Additional costs for snapshots in S3
  27. 27. EBS != hard drive• Slow for time critical tasks• Bottleneck for Lucene indexing and DB – Use parallel EBS volumes – Separate data storage between EC2 instances – Don’t use RDS• Limited to 1TB – Virtual RAID – Data distribution
  28. 28. S3 quick introduction $.055 - $.125 per GB month storage $.05 - $.12 per GB data transfer
  29. 29. Load balancing withJMS or AMQP• JMS and AMQP are very scalable and flexible• Costs are less than SQS (Simple Queue Service)• Configurable load balancing
  30. 30. Deployment options
  31. 31. Self-distributed system• Host manager works via Java AWS API• Centralized deployment• Code distribution to EC2 hosts
  32. 32. Reduce coupling to AWS
  33. 33. Conclusions• AWS is great for startups or research projects• AWS increases development speed in times• It is easy to create scalable and highly available Java application on AWS• Development, testing and production may be done on AWS• AWS save development costs• AWS support Agile developmernt approaches
  34. 34. Useful books
  35. 35. @xpinjection