Download as ODP, PPTX
More Related Content
Similar to Bostonrb Amazon Talk
![[Jun AWS 201] Technical Workshop](https://cdn.slidesharecdn.com/ss_thumbnails/aws201general201tworkshop-130708040331-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Bostonrb Amazon Talk
- 1. Cloud Computing featuringAmazon Web Services an unofficial presentation
- 2. I am... BrianKaney from Vermonster . We are a 3 person shop in the Back Bay. We've been around for over 8 years and mostly do ruby, perl and python stuff. I do not work for Amazon. I don't know every Amazon offering – maybe you can help ?
- 3. Warning! There arelots of acronyms and abbreviations. I'll be bouncing from CLI, Web management console(s) and ruby code (and slides) This has been (somewhat) rehearsed but things happen... but at least you didn't have to pay and there's beer afterward.
- 4. Agenda TODO: Cramall this into 60 mins. Intro
- 5.
- 6.
- 7.
- 8.
- 9.
- 10. Deploy / SCM( System Configuration Management )
- 11.
- 12.
- 13. Agenda TODO: Cramall this into 60 mins. Intro
- 14.
- 15.
- 16.
- 17.
- 18.
- 19. Deploy / SCM( System Configuration Management )
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.
- 29. Intro: Why Cloud?Temporary Events Unpredictability! Scalability Low Barrier to Entry Redundancy Batch Process
- 30. Intro: Uncontroversial PredictionCloud computing is no longer vaporware and will continue growth .
- 31. Intro: Who Clouds?Web Applications
- 32.
- 33. Biotech / Modeling/ Scientific Community
- 34.
- 35. Who else hereclouds?
- 36. Intro: Where canI get one? Engine Yard (Amazon)
- 37. Rightscale (Amazon andothers)
- 38.
- 39.
- 40.
- 41.
- 42.
- 43. Intro: Where canI get one? Engine Yard (Amazon)
- 44. Rightscale (Amazon andothers)
- 45.
- 46.
- 47.
- 48.
- 49.
- 50. Intro: Amazon AWSCurrently... The most mature
- 51. The most fullyfeatured
- 52.
- 53. And the onlyone I've really used
- 54. Intro: How muchdoes it Cost? Minimum hour increments
- 55. Min $0.10/hr ($70/month/instance)
- 56. Max $0.80/hr ($560/month/instance)
- 57.
- 58. Reservations 1/3 thehourly price ($200 - $2000 per year)
- 59. NOTE: See aws.amazon.comfor official pricing
- 60.
- 61.
- 62.
- 63.
- 64.
- 65.
- 66. Deploy / SCM( System Configuration Management )
- 67.
- 68.
- 69. Fundamentals: Amazon APIIt is fairly consistent – REST or WSDL/SOAP
- 70. Security x509 keypairs for API Access
- 71. Separate keypairs installedon the instance Public API
- 72. Instance API vialink-local address
- 73. Fundamentals: Ruby andAmazon aws-s3
- 74.
- 75.
- 76. Tons of others (http://github.com/search?type=Repositories&language=rb&q=amazon)
- 77. Fundamentals: Ruby andAmazon aws-s3
- 78.
- 79.
- 80. Tons of others (http://github.com/search?type=Repositories&language=rb&q=amazon)
- 81. Fundamentals: EC2 ElasticCompute Cloud
- 82.
- 83. 6 sizes – c1.medium best bang for buck
- 84.
- 85. Availability Zones (different for everyone )
- 86. Launch from AMIs( Amazon Machine Images )
- 87. Fundamentals: Launch anInstance!
- 88. Fundamentals: AMIs Typicalto find an existing, customize and then repackage
- 89.
- 90. Well-maintained Ubuntu onesat ( http://alestic.com )
- 91. Consider re-bundle aspart of deployment
- 92. Fundamentals: Let's bundle !
- 93.
- 94.
- 95.
- 96. DNS mapping iscool
- 97. Pretty cheap andwidely used – early offering
- 98. Fundamentals: EBS ElasticBlock Storage
- 99.
- 100.
- 101.
- 102.
- 103.
- 104.
- 105. Written in Erlang(rumors?)
- 106. “Offline” M/DB open-sourceclone (http://www.mgateway.com/mdb.html)
- 107. I haven't usedSDB... anyone using it?
- 108. Fundamentals: SQS SimpleQueue Service
- 109.
- 110. v2 has messagelocking ( recommended )
- 111. Order is notFIFO – but you can provide sequence metadata
- 112. Up to 8kin size
- 113. Fundamentals: Create SQSMessages
- 114.
- 115.
- 116.
- 117.
- 118.
- 119.
- 120. Deploy / SCM( System Configuration Management )
- 121.
- 122.
- 123. ELB/AS: A ProposalI have a random idea: Let's launch an automatically-scalable , load-balanced cloud running a trivial sinatra application!
- 124. ELB/AS: Loadbalance NEW Start with an Elastic LoadBalancer
- 125.
- 126. Balanced across allavailability zones
- 127. Create a realCNAME in real life
- 128. ELB/AS: AutoScaling NEW Create Groups of Instances
- 129. Launch Configuration foreach Group
- 130. One or manyTriggers create Scaling Activities ( add or remove instances ) Period – how often to measure
- 131. Breach Duration – how long the breach is affected
- 132. ELB/AS: AutoScaling ConceptsTrigger 1 Trigger 2 Trigger 3 Launch Config
- 133.
- 134.
- 135.
- 136.
- 137.
- 138.
- 139.
- 140. Deploy / SCM( System Configuration Management )
- 141.
- 142.
- 143. MapReduce Design pattern,coined by Google (?)
- 144.
- 145. For big datasets
- 146.
- 147.
- 148.
- 149. MapReduce: Theory First,map chunks up the data for processing
- 150. Then reduce combinesbased on keys
- 151. Remember math class:map(k, v) -> <k' v'> reduce(k', <v'>) -> <k',v'> Distributed key-stores … nosql.
- 152. MapReduce: EMR ImplementationSTDIN and STDOUT are the interfaces
- 153. Key - valueseparated by “:\t”
- 154. Output of map and reduce should be similar.
- 155. JAR, Streaming, Pig or Hive
- 156. MapReduce: EMR ProposalI have another completely random idea: Let's process a data.gov feed of earthquake data to see the most seismically volatile regions using Elastic Map Reduce !
- 157. MapReduce: While weare Waiting... Minimum of 30 seconds to start a job.
- 158. I don't havetime for that
- 159. Let's look atsome monitoring in the meantime.
- 160.
- 161.
- 162.
- 163.
- 164.
- 165.
- 166. Deploy / SCM( System Configuration Management )
- 167.
- 168.
- 169. Monitoring NEWCloud Watch
- 170. API provides aggregation( sweet! )
- 171. Amazon has anice UI
- 172. But account managementis crappy
- 173. We can writesome code for that!
- 174. Monitoring: Code Startwith a CLI: ec2-monitor-instances <instance-id> RightScale gem and a rake task
- 175. Then use someHTML5 canvas action Thanks to a jQuery visualize plugin by Filament!
- 176. Our Map-Reduce Shouldbe Done!
- 177.
- 178.
- 179.
- 180.
- 181.
- 182.
- 183. Deploy / SCM( System Configuration Management )
- 184.
- 185.
- 186. Other Services CloudFront NEWISH
- 187.
- 188.
- 189. Payment System (DevPay,Flexible Payments)
- 190. Load up atruck and physically ship your data to Amazon
- 191.
- 192.
- 193.
- 194.
- 195.
- 196.
- 197.
- 198. Deploy / SCM( System Configuration Management )
- 199.
- 200.
- 201. Deployment (opinionated) Usea global registry – S3
- 202.
- 203. EC2-on-Rails (shouldn'tit be rails-on-ec2?)
- 204.
- 205.
- 206.
- 207.
- 208.
- 209. Deployment v2 (alsoopinionated) That's fine for a handful of instances
- 210. But direct sshdoes not scale (try deploying to 100, 500 or 1,000 instances)
- 211. Vertebra (dead? usesXMPP)
- 212. Build one ona messaging queue? Use Nanite?
- 213.
- 214.
- 215.
- 216.
- 217.
- 218.
- 219. Deploy / SCM( System Configuration Management )
- 220.
- 221.
- 222. Architecture Redux Stateless– like the web and also like our ruby “best-practices” development.
- 223.
- 224. Messaging instead offorking/backgrounding (AMPQ)
- 225.
- 226.
- 227. Architecture Redux Weneed some state-full pieces
- 228.
- 229. But we canif we drink the kool-aid! SQS S3 SDB LoadBalancer EBS
- 230. Architecture Redux VendorLock-in
- 231. Predetermined technology (SBD.. I'd rather have Postgres )
- 232. But we areagile, so less of an issue.
- 233. I can't backthis up statistically, but my guess is most are using a hybrid architecture.
- 234. What about ExitingRails Applications? We are fairly stateless to begin with
- 235. Configuration Management Inventorygems and plugins
- 236. External libraries SharedStorage Shared NEW EBS / S3 Messaging ( SQS / AMPQ, opposed to forking/backgrounding )
- 237. Log files (Shared EBS helps )
- 238. Consider a ThirdParty Simplify Architecture, Configuration Management and Deployment
- 239.
- 240.
- 241.
- 242. Your Friendly NeighborhoodConsultants
- 243. Thanks! Brian Kaney– twitter.com/ bkaney
- 244.
- 245. Vermonster – http://www.vermonster.com