SD Forum Java SIG - Running Java Applications On Amazon EC2
Upcoming SlideShare
Loading in...5
×
 

SD Forum Java SIG - Running Java Applications On Amazon EC2

on

  • 8,131 views

The Amazon Elastic Compute Cloud (EC2) is ideally suited to running Java applications. It lets you develop using standard Java software packages such as Tomcat and MySQL and rapidly deploy ...

The Amazon Elastic Compute Cloud (EC2) is ideally suited to running Java applications. It lets you develop using standard Java software packages such as Tomcat and MySQL and rapidly deploy applications on servers that are provisioned and managed via a web services API. However, because it is a cloud, some aspects of EC2 are very different than a traditional, physical computing environment. In this session you will learn about those differences and how they impact how you handle security, networking, storage and availability. We describe how to use EC2 and the other Amazon web services to develop and deploy Java applications. You will learn how to use EC2 availability zones to deploy highly available applications. We also discuss how to architect secure applications for Amazon EC2.

Statistics

Views

Total Views
8,131
Views on SlideShare
7,989
Embed Views
142

Actions

Likes
1
Downloads
72
Comments
1

5 Embeds 142

http://www.chrisrichardson.net 115
http://www.cloud24by7.com 12
http://www.slideshare.net 11
http://chrisrichardson.net 3
http://cloudtoolsinc.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • i need a presentation on java application generator project i.e., it sh'd specify the UML diagrams and DFD's and steps included in developing new java applications if u find ,forward to me
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

SD Forum Java SIG - Running Java Applications On Amazon EC2 SD Forum Java SIG - Running Java Applications On Amazon EC2 Presentation Transcript

  • Running Java and Grails applications on Amazon EC2 Chris Richardson Author of POJOs in Action Founder of Cloud Tools and Cloud Foundry y Chris Richardson Consulting, Inc www.chrisrichardson.net
  • Overall presentation g p goal Show how to deploy Java and Grails applications on Amazon Elastic Compute Cloud Slide 2 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • About Chris Grew up in England and live in Oakland, CA • Over 20+ years of software development • experience including 12 years of Java Author of POJOs in Action • Speaker at JavaOne, SpringOne, NFJS, • JavaPolis, Spring Experience, etc. Chair of the eBIG Java SIG in Oakland • (www.ebig.org) Run the Groovy/Grails meetup • (http://java.meetup.com/161) Run a consulting and training company that u co su t g a d t a g co pa y t at • helps organizations reduce development costs and increase effectiveness Founder of Cloud Tools, an open-source project • for deploying Java applications on Amazon EC2: http://code.google.com/p/cloudtools http://code google com/p/cloudtools Founder of a startup that provides outsourced, • automated, and Java-centric datacenter management on the cloud: www.cloudfoundry.com y Slide 3 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Agenda g Amazon-style cloud computing Using Amazon EC2 Deploying on Amazon EC2 pyg Programming with AWS Slide 4 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Power generation g Past Present Slide 5 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Computing has come a long way p g g y Past Present www.computermuseum.org.uk www.dell.com de co Slide 6 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Yet we rarely have enough hardware y g Can we afford the production hardware? Do we know how much to buy? How long does it take to buy and install? Can we afford a test lab? Who is going to set it up and take care of it? Slide 7 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Cloud computing p g A pool of highly scalable, abstracted infrastructure that hosts your application, and is billed by consumption p By James Staten of Forrester Research AND is managed via a web services API me Slide 8 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Amazon-Style Cloud Computing y p g Simple Q Si l Queue SService i Simple DB Si l (SQS) (name/attribute pairs) Elastic Compute Cloud Pay per ( (EC2) ) use services Simple Storage Service managedd (S3) Elastic Block Store (EBS - SAN) by CloudFront Amazon (content delivery) Slide 9 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Sign up g p Login using your existing Amazon account Select the web services you want to use Only takes a few minutes But B t can sometimes ti be confusing: various ids, keys, certificates etc Slide 10 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Make web service calls… <RunInstancesResponse> <reservationId>r-60907709</reservationId> <ownerId>556666664445</ownerId> … <instancesSet> <item> <instanceId>i-4ef21327</instanceId> <imageId>ami-3795705e</imageId> <instanceState> https://ec2.amazonaws.com? <code>0</code> Action=RunInstances <name>pending</name> &ImageId=ami-3795705e </instanceState> &MaxCount=1 <placement> &MinCount=1 <availabilityZone>us-east-1b</availabilityZone> … </placement> <dnsName/> <reason/> / <keyName>gsg-keypair</keyName> <amiLaunchIndex>0</amiLaunchIndex> </item> </instancesSet> / </RunInstancesResponse> Slide 11 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • … a few minutes later cer@arrakis ~ $ ssh … root@ec2-67-202-41-150.compute-1.amazonaws.com Last login: Sun Dec 30 18:54:43 2007 from 71.131.29.181 [root@domU-12-31-36-00-38-23:~] Slide 12 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Deploying a web application on EC2 pyg pp Tomcat Server MySQL (instance 2) DB (Slave) HTTP(S) (instance 5) Web Browser Apache Server MySQL (instance 1) DB (Master) (instance 4) Tomcat Server MySQL EBS Volume (instance 3) DB (Slave) (instance 6) S3 Slide 13 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Pay per use computing yp p g Virtual Compute 32/ Memory Storage $/hr Cores Units 64 ** /core* Bit Small 1 1 32 bit 1.7G 160G 0.10 High High- 2 2.5 25 32 bit 1 7G 1.7G 350G 0.20 0 20 CPU Medium Large ag 2 2 6b 64 bit 7.5G 5G 850G 00 0.40 Extra 4 2 64 bit 15G 1690G 0.80 Large High- High 8 2.5 25 64 bit 7G 1690G 0.80 0 80 CPU XL * EC2 Compute Unit = 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor ** Windows more expensive, external bandwidth: $0.10-0.18/Gbyte Slide 14 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Operating systems p gy Use Amazon provided Machine Image (AMI) 32/64 bit 32/64-bit Fedora Core 4/6/8 Windows Server 2003 ($0.125-$2/hour) Optional SQL Server Standard ($1.10- 3.20/hour) Many 3rd parties have public AMIs Various Linux distributions E.g. Redhat, E g Redhat RightScale Sun provides OpenSolaris Build your own AMI: Install applications starting with existing AMI and save new AMI Create an AMI from scratch Slide 15 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Using AWS in y g your application pp S3 - Store media etc in S3 SQS - messaging between loosely coupled components SimpleDB – alternative to RDBMS CloudFront – to distribute content Using these APIs Couples your application to AWS But using them is optional Slide 16 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Developing on EC2 pg Immediate access to many servers Simplified setup Great for testing g Slide 17 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Deploying on Amazon EC2 – startups/small businesses p/ Some VCs require it Get up and running quickly Validate your business idea without: y Upfront costs Long-term financial commitment g Scale up/down with load Reduces the risk of a success catastrophe Slide 18 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Deploying on Amazon EC2 – enterprises p No need to wait for corporate IT In some companies it can take 2 months to acquire hardware Requires a long-term financial commitment, commitment upfront costs Use for short-term projects, e.g. Websites for marketing campaigns g pg New York Times style projects Use for applications that have fluctuating loads e.g. loads, e g heavily used once a week, once a month Slide 19 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Example – beer on the cloud p Grails application Short-term marketing campaign site Fluctuating load Sat/Sun 4 servers Mon Fri Mon-Fri 1 server Slide 20 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Agenda g Amazon-style cloud computing Using Amazon EC2 Deploying on Amazon EC2 pyg Programming with AWS Slide 21 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • EC2 API and Tools Amazon provided CLI tools CLI equivalents of APIs q AMI creation tools AWS CLI tools from Tim Kay CLI for S3 and EC2 Alternatives to Amazon CLI tools AWS Console Very slick y ElasticFox Awesome Firefox plugin Launch and manage instances S3 Organizer Firefox plugin Manipulate S3 buckets and objects … Slide 22 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • AWS Management Console g Slide 23 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Firefox plugins pg ElasticFox S3 Organizer Slide 24 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Cloud Tools Open-source project 32 and 64 bit AMIs Cent OS 5.10 Apache/Tomcat/MySQL/JMeter/JetS3t installed EC2Deploy framework EC2D l f k Extensible, object-oriented Launches instances Configures Tomcat, MySQL, Apache Deploys web applications •Quicker Runs Jmeter tests deployment •More accurate Written in Groovy configuration Maven and Grails plugins Quick and easy deployment to EC2 Slide 25 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Maven and Grails plugins pg mvn cloudtools:deploy py grails cloud-tools-deploy OR Slide 26 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Extensible provisioning and management p g platform Built using Domain Driven Design Main extension points: New IaaS clouds New i f N infrastructure components/servers t t t/ Implement interfaces or define subclasses Define methods for deploy, start, stop, … Write the scripts Recently added support for Spring dm server and eXo Portal Server 4/6/2009 27 Copyright (c) 2009. Chris Richardson Consulting Inc. Confidential
  • Cloud Foundry y Slide 28 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Agenda g Amazon-style cloud computing Using Amazon EC2 Deploying on Amazon EC2 pyg The basics Running the web tier g Deploying a database Handling security High availability Programming with AWS g g Slide 29 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Issues with AWS Cloud Computing Survey: IT Leaders See Big Promise, Have Big Security Questions Security: Runs HIPAA compliant apps BUT Lack of PCI compliance Discomfort with sending g customer data to a 3rd party Technology: Not yet suitable for extremely large relational databases Lack of very large machines, e.g. 64G memory Lack of multicast and multiple IP addresses Financials: Cost of bandwidth Steady state costs > your own hardware hd www.cio.com/article/455832/Cloud_Computing_Survey_IT_Leaders_See_Big_Promise_Have_Big_Security_Questions Slide 30 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Cost issues Running larger servers 24 x 7 looks expensive ( g $0.80/hr, $560/month) (e.g. $ / ,$ / ) BUT when owning your own hardware Lack of elasticity Long procurement time Must buy for the estimated peak load Must buy redundant hardware Risk of a success catastrophe Cost Electricity ($0.07-$0.30 / kWh), cooling, space System administration costs Management overhead Slide 31 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Starter website - $ w w w .a c m e .c o m Low cost - $72/month E la s tic IP A E C 2 In s ta n c e Elastic - load increases ⇒ expand in a few minutes di f it Apache Available –instance crashes ⇒ replace in a few minutes p T om cat M ySQ L E B S V o lu m e Slide 32 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Higher capacity website - $$ g p y www.acme.com www acme com Low cost - > ~$216/month (1 Elastic IP or more Tomcats, 0 or more Slaves) Apache Elastic - load changes ⇒ quickly expand/subtract Tomcats with no downtime Tomcat Tomcat Available –instance crashes ⇒ replace in a few minutes MySQL MySql (slave) (Master) EBS Volume Slide 33 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Batch processing architecture p g e.g. e g media transcoding Slide 34 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Easy upgrades y pg Clone production environment Make read-only or turn off Snapshot EBS volumes and create new volumes l Apply upgrades to clone Test clone Move elastic IP addresses to clone Terminate old instances once you are sure that everything works Slide 35 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Agenda g Amazon-style cloud computing Using Amazon EC2 Deploying on Amazon EC2 pyg The basics Running the web tier g Deploying a database Handling security High availability Programming with AWS g g Slide 36 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • No hardware load balancing g Coming in 2009 Use software load balancer Apache HAProxy … Slide 37 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Elastic IP addresses Instance IP addresses are dynamically allocated on start-up start up Does not work well for publicly accessible services, e.g. a website Elastic IP addresses: Statically allocated public IP addresses Associated with your account Attached to an instance (e g public facing web (e.g. server) = it's public IP address You configure DNS to resolve to the elastic IP address Pricing: Non-attached Elastic IP address - $0.01/hour $0.10 $0 10 per remap (if > 100 in a month) Slide 38 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Elastic IP address operations p Operation Parameters XML document DescribeAddresses PublicIp.n (optional) List of IP addresses and associated instance id AllocateAddress - Public IP address Release Address Public Ip address - AssociateAddress InstanceId, InstanceId Public IP - Address DisasssociateAddress Public IP Address - Slide 39 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Agenda g Amazon-style cloud computing Using Amazon EC2 Deploying on Amazon EC2 pyg The basics Running the web tier g Deploying a database Handling security High availability Programming with AWS g g Slide 40 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Elastic Block Storage g Local storage is ephemeral Mountable storage volumes M t bl t l quot;On-demand SANquot; Size: 1 GB to 1 TB Mount on a single instance Create snapshots p Stored in S3 Create new volumes from the snapshot Cost: C $0.10/GByte/month $0.10 $0 10 per 1 million I/O requests Slide 41 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Using EBS Volumes g AWS: CreateVolume Size=50G AttachVolume InstanceId=… Device=/dev/sdh mkfs.xfs mkfs xfs /dev/sdh echo quot;/dev/sdh /vol xfs noatime 0 0quot; >> /etc/fstab mkdir /vol mount /vol mkdir /vol/lib /vol/log mv /var/lib/mysql /vol/lib [mysql.server] user=mysql basedir=/vol/lib basedir /vol/lib Slide 42 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Backing up your database g py mysqldump --add-drop-database --databases foo | gzip > backup.sql.gz now=`date +%d%m%y_%H%M` aws put $bucket/${object}_${now}.sql.gz backup.sql.gz aws copy $bucket/${object}_latest $bucket/${object}_${now}.sql.gz FLUSH TABLES WITH READ LOCK SHOW MASTER STATUS xfs_freeze -f /vol # AWS WS: CreateSnapshot xfs_freeze u xfs freeze -u /vol UNLOCK TABLES Slide 43 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Agenda g Amazon-style cloud computing Using Amazon EC2 Deploying on Amazon EC2 pyg The basics Running the web tier g Deploying a database Handling security High availability Programming with AWS g g Slide 44 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Security benefits of cloud computing p g Leverages the world class security techniques of amazon.com Cloud infrastructure enables: Unlimited logging Ability to test changes on a clone Clone servers and volumes for forensic analysis Slide 45 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • The usual security best practices y p Turn off unused services File ownership and permissions Disabling password based ssh login gp g Standard Linux, Apache, Tomcat and MySQL best practices yQ p Slide 46 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Network security y Cannot sniff traffic for other instances Use EC2 firewall – aka. security groups Consider encrypting network traffic Limit SSH access to only your location yy Slide 47 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Security Groups y p Named set of firewall rules associated with your account An instance Belongs to one or more security groups Defaults to “default” security group g Permits inbound traffic ?Action=RunInstances Protocol: tcp, udp &SecurityGroup.1=g1 Range of ports &SecurityGroup.2=g2 From: Anywhere – specific port range An IP address (range) – specific port range Another group - all ports When you first Common usage signup don’t Port 80 (http)/443 (https) – anywhere forget to enable Port 22 (ssh) – just from your location SSH traffic ff Slide 48 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Using security groups g yg p Slide 49 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Use a software firewall E.g. iptables In addition to security groups Security Group: Tomcat Servers are only accessible from Apache Server iptables: Tomcat servers only allow port 22 and port 8009 (AJP) po t Slide 50 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Storage security g y Amazon wipes disks so one customer cannot see another’s data But You don’t know where it is Amazon could be subpoena’d Consider encrypting data Encrypted file systems Encrypting sensitive data in DB Encrypting backups in S3 Slide 51 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Agenda g Amazon-style cloud computing Using Amazon EC2 Deploying on Amazon EC2 pyg The basics Running the web tier g Deploying a database Handling security High availability Programming with AWS g g Slide 52 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Deploying highly available applications pp AWS has had very well publicized outages BUT… Is internal IT really any better? In reality: AWS is (more) reliable y ( ) Don’t forget: You are not responsible for the hardware Instance fails ⇒ Launch a new one in a few minutes Slide 53 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • But once in a blue moon From: Amazon EC2 Notification ec2-notification@amazon.com Subject: Notice: Degraded Amazon EC2 Instance To: XXXXX@yahoo com XXXXX@yahoo.com Date: Friday, January 23, 2009, 5:54 AM Hello, We have noticed that one or more of your instances are running on a host degraded due to hardware failure failure. i-5e0b8b34 The risk of your instances failing is increased at this point. We cannot determine the health of any applications running on the instances. We recommend that you launch replacement instances and start migrating to them. Feel free to terminate the instances with the ec2-terminate-instance API when you are done with them. Let us know if you have any questions questions. Sincerely, The Amazon EC2 Team Slide 54 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Can't migrate internal IP addresses g Instance has one fixed, internal IP address Using Elastic IP = $ Therefore Handling active/standby failover is difficult: E.g. Cannot migrate IP address of failed database to standby database Have your own host names Update /etc/hosts Run DNS server Slide 55 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • No multicast for resource discovery y Prevents the use of standard clustered resource discovery E.g. JGroups etc Use a registry: Database SimpleDB Security groups … Slide 56 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Regions and availability zones g y By default, your database us- eu- master and slave could run on the same physical host! east-1 west-1 Regions - geographically dispersed locations p Availability zone - us-east- engineered to be insulated eu- from failure in other zones 1a, west 1a west-1a Specify availability zone p y y when launching instances us-east- SLA with 99.95% availability with multiple 1b availability zones eu- You pay for inter-zone us-east- network traffic west-1b 1c Slide 57 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Amazon EC2 SLA* 99.95% availability if you are using >1 availability zone Availability Instances have external connectivity You can launch new instances Service credit for not meeting SLA * Read the small print Slide 58 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Regions and Availability Zones API g y Operation Parameters XML document DescribeRegions Region.n (optional) List of region names and urls DescribeAvailabilityZones ZoneName n ZoneName.n List of availability zones and state https://<region>.ec2.amazonaws.com? Action=RunInstances &Placement.AvailabilityZone=<availabilityZone> https://ec2.amazonaws.com? Action=RunInstances &Placement.AvailabilityZone=<availabilityZone> &Placement AvailabilityZone <availabilityZone> Slide 59 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Highly available - $$$ gy www.acme.com Higher cost - > ~$ $ 360/month (2 Apaches, 2 MySqls, 1 or more Tomcats, 0 Elastic IP B or more Slaves) Elastic IP A Availability Zone A Availability Zone B Elastic load h El ti - l d changes ⇒ quickly expand/subtract Apache Apache Tomcats with no downtime Available – No SPOF, instance , crashes ⇒ replace in a few Tomcat Tomcat Tomcat Tomcat minutes MySQL MySQL (Master 2) (Master 1) EBS Volume EBS Volume Slide 60 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Agenda g Amazon-style cloud computing Using Amazon EC2 Deploying on Amazon EC2 pyg Programming with AWS Slide 61 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Using AWS in y g your application pp Access instance meta data Simple Storage Service (S3) Stores blobs of data Eg. Photo sharing website Store media Hand out URLs to S3 objects Simple Queue Service (SQS) Hosted queue-based messaging system q ggy Alternative to JMS Loosely coupling between systems SimpleDB Schema-less non-relational database Store data sets Execute queries Slide 62 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Eventual consistency y AWS is distributed Data is replicated among many nodes Replication takes time Updates eventually appear Why? CAP theorem by Brewer Pick two: consistency, availability, partitioning Example: S3 – a GET might not see a PUT SQS – reading from a queue might not retrieve recently added messages … Slide 63 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Instance meta data Instance can find out: Information about itself f b lf User data supplied by user at launch time Enables a generic AMI to customize itself g dynamically Available data includes: user data user-data security-groups public-hostname placement/availability zone placement/availability-zone … curl http://169.254.169.254/2008-12-01/meta-data/<<data type>> Slide 64 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Amazon Simple Storage Service ( ) p g (S3) Flat storage model consisting of buckets and objects Bucket has a name, e.g. <AccessKey>.<name> name e g <AccessKey> <name> contains objects Objects j Has a key, e.g. mypicture.jpg Stores 1 byte - 5G Simulating a hierarchical file-system Si l ti hi hi l fil t Object key can look like a path ☺ presentations/february09/aws.ppt presentations/february09/aws ppt Slide 65 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • S3 REST API PUT / HTTP/1.1 Create a bucket Host: <BucketName>.s3.amazonaws.com … PUT /<ObjectName> HTTP/1 1 HTTP/1.1 Host: <BucketName>.s3.amazonaws.com … Create an item in a bucket …Bytes… GET /<ObjectName> HTTP/1.1 Host: <BucketName>.s3.amazonaws.com Download an item … DELETE /<ObjectName> HTTP/1.1 Host: <BucketName>.s3.amazonaws.com … Delete an item Slide 66 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Amazon CloudFront Content delivery network Original content stored in S3 bucket Register publically accessible bucket with Cl dF ith CloudFront ⇒ unique d t i domaini name (foo1234.cloudfront.net) Content accessed through that domain name is delivered by geographically distributed edge servers http:// foo1234.cloudfront.net/i/bar.jpg ⇒ <BucketName>/i/bar.jpg k / /b Slide 67 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • SimpleDB model p Domain id description color Has a name 123 jeans blue, Contains items black Item: 456 shoes red, white Has a name Has one or more attributes Attribute: Has a name Has one or more values Slide 68 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Simple DB model p It’s not a relational database No joins Eventual consistency - updates eventually appear No transactions – single item update No locking Limits 100 domains per account 250,000,000 attribute name-value pairs per domain 256 attribute name-value pairs per item Queries return … Pricing: Machine utilization: $0 14/hour after first 25 free $0.14/hour hours/month Fees for data transfer in and out (Free for access from EC2) Slide 69 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • SimpleDB Operations p p SOAP and REST API Domains: Create/List/Delete PutAttributes DomainName AttributeName Attribute.N.Name/Attribute.N.Value Attribute.N.Replace – add or replace DeleteAttributes DomainName AttributeName Attribute.N.Name/Attribute.N.Value GetAttributes DomainName AttributeName.n Slide 70 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • SimpleDB Select queries p q “SQL-like” Select query language Various limitations Sort by attribute must appear in where clause and select list … select * Select operation from domainName where SomeAttribute > 2 SelectExpression order by SomeAttribute limit 10 NextToken Slide 71 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • SimpleDB custom queries p q Query [ SomeAttribute [‘SomeAttribute’ > 2] Domain sort ‘SomeAttribute’ QueryExpresssion Pagination with: MaxNumberOfItems/NextToken Returns It Rt ItemNames N QueryWithAttributes Adds AttributeName.n ( Add A ib N (optional) i l) Returns values Slide 72 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Using SimpleDB g p Replace joins by denormalizing/duplicating data E.g. Duplicate child data in parent in parent-child relationship li hi E.g. http://blog.adaptiveblue.com/?p=1145 People-Interaction-Thing (1-N N-1) Store Interaction in People and in Thing Parallelize SimpleDB requests An application should/could issue multiple requests in parallel Partition d data across multiple d l l domains E.g. People1, People2 Improves performance Use SimpleDB when: You can tolerate inconsistencies You don’t need transactions i.e. bad for banking but good for social network, read intensive d t it i data Slide 73 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Amazon SQS Q Queues: As many as you want Unlimited size Messages deleted after 4 days AWS might delete queues that are idle for > 30 days Message are to 8Kb (store binary and larger messages in S3 or SimpleDB) Semantics of distributed queuing Order is not guaranteed At-least once l ReceiveMessage returns messages from a subset of servers, e.g. possibly no messages Pricing: g $0.000001 per Request $0.100 per GB – all data transfer in $0.170-0.100 per GB – data transfer out Slide 74 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • SQS API details Q SOAP API only Managing queries Create/List/Delete Queues Sending a Message SendMessage Processing a message ReceiveMessage DeleteMessage e ete essage Visibility timeout: a received message that is not deleted within the timeout will reappear Queue attributes SetQueueAttributes/GetQueueAttributes ApproximateNumberOfMessages VisibilityTimeout Slide 75 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Java libraries for AWS JetS3t Rich API for accessing S3 j jets3t.dev.java.net/ j Typica API for SQS, EC2, SimpleDB code.google.com/p/ty pica SimpleJPA Subset of JPA on Simple DB code.google.com/p/si mplejpa Slide 76 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Summary y Amazon-style cloud computing provides Immediate access to a scalable infrastructure Pay as you go – no upfront P f t investment/commitment required Easily scale up/down Optional AWS services Slide 77 Copyright (c) 2009 Chris Richardson. All rights reserved.
  • Final thoughts g Download or contribute to Cloud Tools today : y www.cloudtools.org Checkout Cloud Foundry: www.cloudfoundry.com www cloudfoundry com Buy my book ☺ Send email: chris@chrisrichardson.net Visit my website: www.chrisrichardson.net Talk to me about consulting and training Phone: 510 904 9832 Slide 78 Copyright (c) 2009 Chris Richardson. All rights reserved.