In the clouds no one can hear you scream...

8,102 views

Published on

Presentation to http://www.devevening.co.uk on 16th April - covering .Net integration with Amazon AWS (S3 and EC2), Mosso, GoGrid, Google Apps Engine, and an introduction to Azure and to SQL Data Services - a bit rushed on the night! :)

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
8,102
On SlideShare
0
From Embeds
0
Number of Embeds
2,183
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

In the clouds no one can hear you scream...

  1. 1. In the cloud no-one can hear you scream… Stuart Lodge [email_address] @slodge http://slodge.blogspot.com http://www.cirrious.com
  2. 2. What is “the cloud”? <ul><li>The Cloud - Processing power and storage considered as consumables. </li></ul><ul><li>Software As A Service - Higher level facilities – the idea of paying for software like paying for electricity… </li></ul>
  3. 3. Why should I care? <ul><li>Scalability </li></ul><ul><li>Cost </li></ul><ul><li>Easier life </li></ul>
  4. 4. AWS <ul><li>The Pioneers </li></ul><ul><ul><li>Elastic Compute Cloud </li></ul></ul><ul><ul><li>Simple DB </li></ul></ul><ul><ul><li>Simple Storage – S3 </li></ul></ul><ul><ul><li>CloudFront </li></ul></ul><ul><ul><li>Simple Queue Service </li></ul></ul><ul><ul><li>Map Reduce </li></ul></ul><ul><li>3 years old – most established by far </li></ul>
  5. 5. S3 – Simple Storage Service <ul><li>“ Buckets” for storing “files” </li></ul><ul><li>File size 1B to 5GB </li></ul><ul><li>Secure authentication (plus public options) </li></ul><ul><li>REST interface (SOAP and BitTorrent also available) </li></ul><ul><li>Reliable, fast, scalable, inexpensive </li></ul><ul><li>Cost: </li></ul><ul><ul><li>$0.15 per GB stored per month, $0.1 per GB transfer in, $0.17 per GB transfer out </li></ul></ul>
  6. 6. Who would use such a thing? <ul><li>Anyone keen to have a scalable, reliable product but with minimal upfront costs and commitments. </li></ul><ul><li>Startups </li></ul><ul><li>SMEs </li></ul><ul><li>Big Es </li></ul><ul><li>Q1 2008 – Amazon quoted 60,000 customers, most of them not startups. </li></ul><ul><li>Image and video storage common. </li></ul><ul><li>Online backup solutions common. </li></ul>
  7. 7. Using S3 From .Net <ul><li>Sign up for AWS </li></ul><ul><ul><li>UserId and Secret Key </li></ul></ul><ul><li>Download ThreeSharp - http://www.codeplex.com/ThreeSharp </li></ul><ul><li>Use ThreeSharp Form Sample UI to create a bucket </li></ul><ul><li>Write your code… </li></ul><ul><li>(Afterwards additional tools like Cloudberry Explorer might also help!) </li></ul>
  8. 8. Simple AWS S3 code public string AddActivity(Activity_t activity) { Guid g = Guid.NewGuid(); string keyName = g.ToString(&quot;N&quot;); // N is guid as hex with no punctuation :) string activityAsText = TCXParser.TCXParser.ActivityToString(activity);   ThreeSharpWrapper wrapper = new ThreeSharpWrapper(AwsAccessKey, AwsSecretKey); wrapper.AddStringObjectWithCompression(AwsActivityBucketName, keyName, activityAsText);   return keyName; } public Activity_t GetActivity(string activityKeyName) { ThreeSharpWrapper wrapper = new ThreeSharpWrapper(AwsAccessKey, AwsSecretKey); string activityAsText = wrapper.GetStringObjectWithDecompression(AwsActivityBucketName, activityKeyName); return TCXParser.TCXParser.StringToActivity(activityAsText); }
  9. 9. Example S3 Costs <ul><li>RunSaturday </li></ul><ul><li>Data store used with caching on my server </li></ul><ul><ul><li>15GB of xml data – compressed down to 0.6GB – $0.09 </li></ul></ul><ul><ul><li>Data transferred in – 0.448GB - $0.04 </li></ul></ul><ul><ul><li>Data out – 0.431GB - $0.07 </li></ul></ul><ul><ul><li>16000 insert (PUT) requests - $0.16 </li></ul></ul><ul><ul><li>10000 GET requests - $0.01 </li></ul></ul><ul><li>Total cost for March - $0.42 </li></ul>
  10. 10. The clock is ticking…
  11. 11. Cloud Compute Power <ul><li>Amazon EC2 </li></ul><ul><li>Mosso/SliceHost </li></ul><ul><li>RightScale and others… </li></ul><ul><li>GoGrid </li></ul><ul><li>Virtual servers… on demand processing. </li></ul><ul><li>Key point - you can turn the servers off! </li></ul>
  12. 12. Using GoGrid, Amazon,… <ul><li>GoGrid and EC2 give you access to virtual servers – whole Win2k3 or Win2k8 (or Linux) servers running as virtual PCs on real servers. </li></ul><ul><li>Very quick and easy to set up or tear down additional servers. </li></ul><ul><li>I chose GoGrid based on cost – especially MS SQL software license costs. </li></ul>
  13. 13. GoGrid Servers – starting new…
  14. 14. A GoGrid Server…
  15. 15. The Cloud And Google… <ul><li>Google’s internals provide a “cloud” too </li></ul><ul><ul><li>Big Table </li></ul></ul><ul><ul><li>Google File System </li></ul></ul><ul><ul><li>Map Reduce </li></ul></ul><ul><li>Focused on Scalability. Trading offline processing to give instant online results. Using generic commodity hardware to reduce costs. </li></ul><ul><li>Google Apps Engine – Python, Django, BigTable… and more recently Java. </li></ul>
  16. 16. The Cloud and Microsoft… <ul><li>October PDC 2008 “Windows Azure” announced </li></ul><ul><ul><li>A Marketing term for pretty much everything new and shiny – the Azure Services Platform. </li></ul></ul><ul><ul><li>Azure applications alongside 3 specific storage mechanisms – Blob, Table and Queue. </li></ul></ul><ul><ul><li>More recently… Azure applications and SQL Data Services seem to have gotten much closer… </li></ul></ul>
  17. 17. Azure 2008 - all about scale
  18. 18. Inside “Azure” <ul><li>Web role </li></ul><ul><li>Worker role </li></ul><ul><li>Table Storage </li></ul><ul><li>Blob Storage </li></ul><ul><li>Queue Storage </li></ul>
  19. 19. Web Role <ul><li>ASP.Net application running in medium trust </li></ul><ul><ul><ul><li>Idea is that this is used for a “Front end” </li></ul></ul></ul><ul><ul><ul><li>Number of instances scalable </li></ul></ul></ul><ul><ul><ul><li>CLR, DLR, AJAX, … - all available (.Net 3.5 SP1) </li></ul></ul></ul><ul><ul><ul><li>No access to permanent local file resources (can use some temporary storage if required). </li></ul></ul></ul><ul><ul><ul><li>Full ASP.Net functionality… but you can’t install anything that needs to go into GAC. </li></ul></ul></ul><ul><ul><ul><li>New (coming soon?) native code execution. </li></ul></ul></ul>
  20. 20. Worker Role <ul><li>Batch processing </li></ul><ul><ul><li>Idea is that this is used for Batch Processing – the “backend”. </li></ul></ul><ul><ul><li>Number of instances scalable </li></ul></ul><ul><ul><li>Can read/write to the datastores. </li></ul></ul><ul><ul><li>Can make HTTP and HTTPS requests. </li></ul></ul>
  21. 21. Table Storage <ul><li>Structured storage </li></ul><ul><ul><li>Main storage for Azure “index” information </li></ul></ul><ul><ul><li>Almost (kind of) relational storage </li></ul></ul><ul><ul><li>Rows indexed by type, then within type by two strings - PartitionKey and RowKey </li></ul></ul><ul><ul><li>Lots of data types - </li></ul></ul><ul><ul><li>“ Foreign Keys” – but not really any joining to be seen. </li></ul></ul><ul><ul><li>Access using REST – coupled with LINQ – but forget “ORDER BY” or “GROUP BY” and “WHERE” is also not quick (yet) </li></ul></ul>
  22. 22. Blob Storage <ul><li>File Storage </li></ul><ul><ul><li>Amazon S3? </li></ul></ul><ul><ul><li>REST Interface – wrapped in code </li></ul></ul><ul><ul><li>HTTP Access too </li></ul></ul>
  23. 23. Queue Storage <ul><li>FIFO queues </li></ul><ul><ul><li>Main use seems to be to allow Web Roles to queue up work for Worker Roles. </li></ul></ul>
  24. 24. Deployment tools A portal for deployment. A private staging area as well as a live “production” area. Support for multiple web role instances, multiple worker roles, multiple storage areas. Some analytics Some logging Quite powerful, but Dog slow to use…
  25. 25. www.stacka.com
  26. 26. stacka’s structure <ul><li>Stacka </li></ul><ul><li>Azure WebRole (like a ASP.NET web app) </li></ul><ul><li>Azure WorkerRole (like a .NET command line app) </li></ul><ul><li>Azure Queue Storage </li></ul><ul><li>Worker-Web Role Comms </li></ul><ul><li>need to add more! </li></ul><ul><li>Azure Table Storage </li></ul><ul><li>Membership </li></ul><ul><li>Stacks </li></ul>Azure Blob Storage - Image Storage
  27. 27. stacka’s twin – www.cloud dot net.com
  28. 28. All about scale
  29. 29. March 2009… Azure reinvented… <ul><li>Customer problems… </li></ul><ul><li>Scale isn’t everything. </li></ul><ul><li>Most people don’t want to write YouTube or Twitter. </li></ul><ul><li>Some (most) apps are really hard to write with Azure Table Storage. </li></ul><ul><li>How to get Azure to help the typical web developer? </li></ul><ul><li>Microsoft’s answer: </li></ul><ul><li>SQL Data Services – almost full MS SQL 2008 support. </li></ul>
  30. 30. Azure 2009 Application Structure Web Roles Worker Roles SQL Blob Queue Microsoft’s sales pitch - easier to maintain, easier to scale (but not massively scale in version 1), cheaper to deploy.
  31. 31. Some screaming… <ul><li>AWS – a few hours outage (Feb 2008) </li></ul><ul><li>Azure - 22 hour outage (March 2009) </li></ul><ul><li>GoGrid – a couple of big outages (March 2009) </li></ul><ul><li>Loss of ownership can make it harder – to debug, to optimise, etc.. </li></ul><ul><li>Azure – SQL Services – if you developed on CTP1 then all your code is pretty much wasted! </li></ul>
  32. 32. More Azure examples <ul><li>Issue Tracker – full bug tracker http://www.codeplex.com/azureissuetracker </li></ul><ul><li>Storage Explorer http://azurestorageexplorer.codeplex.com/ </li></ul><ul><li>LifeTracks http://lifetracks.codeplex.com/ </li></ul><ul><li>Blog engine http://blog.smarx.com/ </li></ul><ul><li>ADXStudio CMS http://www.shanmcarthur.net/ </li></ul><ul><li>Bluehoo http://www.bluehoo.com/ </li></ul><ul><li>Twitter apps, DLR (IronPython) code, facebook apps, … - see http://www.clouddotnet.com/ </li></ul>
  33. 33. How does someone get started? <ul><li>Get a key from Microsoft – www.azure.com </li></ul><ul><li>Get the SDK and samples from Microsoft </li></ul><ul><li>Look at and start from the samples. </li></ul><ul><li>Look at the PDC videos. </li></ul><ul><li>For SQL, wait for the next release… this summer </li></ul><ul><li>More help: </li></ul><ul><ul><li>http://blog.smarx.com </li></ul></ul><ul><ul><li>http://social.msdn.microsoft.com/Forums/en-US/windowsazure/threads/ </li></ul></ul>
  34. 34. www.runsaturday.com
  35. 35. runsaturday’s structure G <ul><li>RunSaturday </li></ul><ul><li>Core C# application code </li></ul>SQL Server 2005 - All data storage <ul><li>Facebook Connect </li></ul><ul><li>Membership </li></ul>Nike+, Nokia, … - Data Collection <ul><li>Web Service UI </li></ul><ul><li>Uploads from script and SportTracks </li></ul><ul><li>Main UI </li></ul><ul><li>DotNetNuke 5.0 with custom modules </li></ul><ul><li>IFrame UI </li></ul><ul><li>Blog and other presentation </li></ul><ul><li>Facebook UI </li></ul><ul><li>FBML Presentation </li></ul>Amazon S3 - GPS Data Storage GoGrid Virtual Server(s) Google maps - Map provision
  36. 36. www.clouddotnet.com www.stacka.com www.runsaturday.com slodge.blogspot.com @slodge

×