10 things I’ve learnt In the clouds


Published on

Presentation on some of my recent experiences with Microsoft Azure, Amazon S3, GoGrid and other cloud technologies, especially while developing:
- http://www.runsaturday.com
- http://www.stacka.com
- http://www.clouddotnet.com

I'm presenting this tonight at the London .Net User's group - but thought it would be useful to share more widely!

If you need more info, contact me@slodge.com - please mark your email with No Spam somehow... hopefully it will get through to me.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

10 things I’ve learnt In the clouds

  1. 1. 10 Things I’ve learnt in the cloud Stuart Lodge [email_address] http://slodge.blogspot.com
  2. 2. 1. There’s more than one cloud <ul><li>“ Software As A Service” </li></ul><ul><li>Scalable Storage – blobs, queues, tables – S3, Azure </li></ul><ul><li>Data services </li></ul><ul><li>Membership services – Live, Google, OpenId, Facebook Connect, … </li></ul><ul><li>Virtual application hosts (GAE and Azure) </li></ul><ul><li>Virtual servers and virtual shared hosting (EC2, GoGrid, Mosso, …) </li></ul><ul><li>… </li></ul>
  3. 3. www.stacka.com
  4. 4. stacka’s structure <ul><li>Stacka </li></ul><ul><li>Azure WebRole (like a ASP.NET web app) </li></ul><ul><li>Azure WorkerRole (like a .NET command line app) </li></ul><ul><li>Azure Queue Storage </li></ul><ul><li>Worker-Web Role Comms </li></ul><ul><li>need to add more! </li></ul><ul><li>Azure Table Storage </li></ul><ul><li>Membership </li></ul><ul><li>Stacks </li></ul>Azure Blob Storage - Image Storage
  5. 5. stacka’s twin – www.cloud dot net.com
  6. 6. www.runsaturday.com
  7. 7. runsaturday’s structure G <ul><li>RunSaturday </li></ul><ul><li>Core C# application code </li></ul>SQL Server 2005 - All data storage <ul><li>Facebook Connect </li></ul><ul><li>Membership </li></ul>Nike+, Nokia, … - Data Collection <ul><li>Web Service UI </li></ul><ul><li>Uploads from script and SportTracks </li></ul><ul><li>Main UI </li></ul><ul><li>DotNetNuke 5.0 with custom modules </li></ul><ul><li>IFrame UI </li></ul><ul><li>Blog and other presentation </li></ul><ul><li>Facebook UI </li></ul><ul><li>FBML Presentation </li></ul>Amazon EC2 - GPS Data Storage GoGrid Virtual Server(s) Google maps - Map provision
  8. 8. Azure <ul><li>In ASP.Net terms: </li></ul><ul><li>Medium trust </li></ul><ul><li>No persistent local file storage – but you can use a temp file area. </li></ul><ul><li>You cannot install anything into the GAC – assume all you have is 3.5 SP1 (4.0 “soon”) </li></ul><ul><li>You can have “worker roles” – background tasks! </li></ul><ul><li>In data terms: </li></ul><ul><li>For “structured” storage, you have “tables” – very good at storing huge amounts of data and at retrieving data using primary key – do not think “SQL” though. </li></ul><ul><li>For files you have Blob. </li></ul><ul><li>For messaging you have Queues. </li></ul><ul><li>- There are other services available… </li></ul>
  9. 9. All about scale
  10. 10. 2. Azure - Simple things are not so simple <ul><li>For example counters… </li></ul><ul><ul><li>Potentially thousands of concurrent clients writing to a single resource is a blocking point – so Azure design discourages it. </li></ul></ul><ul><ul><li>There will be patterns evolve – e.g. sharding counters can help. </li></ul></ul><ul><li>Example – keeping track of number of tagged entries in Stacka. </li></ul>
  11. 11. 3. Azure - goodbye to SQL Group By <ul><li>Also goodbye to Min, Max, Count, Average… </li></ul><ul><li>These methods just don’t scale… </li></ul><ul><li>If you want stats you have to work them out and store them </li></ul><ul><li>Google’s Map Reduce as an example – Google Analytics. </li></ul><ul><li>What’s inefficient? </li></ul><ul><li>Example – Stacka– how many stacks has a user uploaded? </li></ul>
  12. 12. 4. Azure - goodbye to transactions. Hello to queues and optimistic updates. <ul><li>For speed of service, Azure tables only support optimistic locking. </li></ul><ul><li>Your design needs to work out how to deal with conflicts. </li></ul><ul><li>Again, new patterns are needed and will evolve. </li></ul><ul><li>Example - stacka – deleting a stack… the cross-table steps involved… possible patterns to solve this… </li></ul>
  13. 13. 5. Azure – get used to waiting <ul><li>The deployment environment, the development environment and the tools and libraries provided by Microsoft are all very advanced and do provide a really rich environment. </li></ul><ul><li>But… it can take a long, long, long time to deploy and put live a deployment on Azure! </li></ul><ul><li>When using the portal, take it *slow* </li></ul>
  14. 14. But this is not complaining… <ul><li>The CTP release is feature rich and it works! </li></ul><ul><li>There are plenty of things you can do: </li></ul><ul><ul><li>Develop on your desktop using all your normal tools. </li></ul></ul><ul><ul><li>Debug use either local SQL Server cloud emulation – or you can develop against real cloud storage </li></ul></ul><ul><ul><li>Use your existing .Net code </li></ul></ul><ul><ul><li>Use AJAX (Atlas) including the Ajax toolkit </li></ul></ul><ul><ul><li>Use both Web and Worker Roles </li></ul></ul><ul><ul><li>There’s first stage analytics – and logging - available </li></ul></ul><ul><ul><li>Use simultaneous production and staging servers – to ease testing of new releases. </li></ul></ul><ul><ul><li>Develop truly scalable solutions </li></ul></ul><ul><ul><li>Develop new ideas, new technologies, new businesses. </li></ul></ul>
  15. 15. Still all about scale
  16. 16. Using GoGrid, Amazon,… <ul><li>Very different to Azure and GAE. </li></ul><ul><li>GoGrid and EC2 give you access to virtual servers – whole Win2k3 or Win2k8 (or Linux) servers running as virtual PCs on real servers. </li></ul><ul><li>Very quick and easy to set up or tear down additional servers. </li></ul><ul><li>I chose GoGrid based on cost. </li></ul>
  17. 17. 6. GoGrid Server – what you get
  18. 18. 7. Using AWS S3 <ul><li>Really scalable storage solution for files (blobs) – same as Azure Blobs – don’t know how pricing will compare? </li></ul><ul><li>ThreeSharp on CodePlex – simple, straight-forward – superb. </li></ul>public string AddActivity(Activity_t activity) { Guid g = Guid.NewGuid(); string keyName = g.ToString(&quot;N&quot;); // N is guid as hex with no punctuation :) string activityAsText = TCXParser.TCXParser.ActivityToString(activity);   ThreeSharpWrapper wrapper = new ThreeSharpWrapper(AwsAccessKey, AwsSecretKey); wrapper.AddStringObjectWithCompression(AwsActivityBucketName, keyName, activityAsText);   return keyName; } public Activity_t GetActivity(string activityKeyName) { ThreeSharpWrapper wrapper = new ThreeSharpWrapper(AwsAccessKey, AwsSecretKey); string activityAsText = wrapper.GetStringObjectWithDecompression(AwsActivityBucketName, activityKeyName); return TCXParser.TCXParser.StringToActivity(activityAsText); }
  19. 19. The clock is ticking…
  20. 20. 8. Facebook Connect <ul><li>Easy way to get users to register on your site. </li></ul><ul><li>Can be integrated with ASP.Net Membership (and with DotNetNuke membership) </li></ul><ul><li>A way to get access to more publicity – to more potential users. </li></ul><ul><li>But… </li></ul><ul><ul><li>facebook is not a “’stable” platform… </li></ul></ul>
  21. 21. 9. DotNetNuke – Look Again <ul><li>Powerful .Net CMS. </li></ul><ul><li>5.0 is 10x faster than I remember 4.0 </li></ul><ul><li>Commercial module availability good. </li></ul><ul><li>Developing modules internally using Linq, 3.5, AJAX, etc are all easy – as easy as developing controls for normal ASP.Net </li></ul><ul><li>100% access to source code (almost) </li></ul><ul><li>Instant tools in your application – like Log Viewer, User Manager, SQL editor, etc </li></ul><ul><li>But… </li></ul><ul><ul><li>Not suitable for Azure… </li></ul></ul>
  22. 22. 10. Microsoft Chart Control <ul><li>Based On Dundas. </li></ul><ul><li>Included in 4.0 </li></ul><ul><li>Simple add-on to 3.5 SP1. </li></ul><ul><li>Powerful. Fast. Good. </li></ul>
  23. 23. www.clouddotnet.com www.stacka.com www.runsaturday.com slodge.blogspot.com …