Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

10 things I’ve learnt In the clouds

3,786 views

Published on

Presentation on some of my recent experiences with Microsoft Azure, Amazon S3, GoGrid and other cloud technologies, especially while developing:
- http://www.runsaturday.com
- http://www.stacka.com
- http://www.clouddotnet.com


I'm presenting this tonight at the London .Net User's group - but thought it would be useful to share more widely!

If you need more info, contact me@slodge.com - please mark your email with No Spam somehow... hopefully it will get through to me.

Published in: Technology
  • Be the first to comment

10 things I’ve learnt In the clouds

  1. 1. 10 Things I’ve learnt in the cloud Stuart Lodge [email_address] http://slodge.blogspot.com
  2. 2. 1. There’s more than one cloud <ul><li>“ Software As A Service” </li></ul><ul><li>Scalable Storage – blobs, queues, tables – S3, Azure </li></ul><ul><li>Data services </li></ul><ul><li>Membership services – Live, Google, OpenId, Facebook Connect, … </li></ul><ul><li>Virtual application hosts (GAE and Azure) </li></ul><ul><li>Virtual servers and virtual shared hosting (EC2, GoGrid, Mosso, …) </li></ul><ul><li>… </li></ul>
  3. 3. www.stacka.com
  4. 4. stacka’s structure <ul><li>Stacka </li></ul><ul><li>Azure WebRole (like a ASP.NET web app) </li></ul><ul><li>Azure WorkerRole (like a .NET command line app) </li></ul><ul><li>Azure Queue Storage </li></ul><ul><li>Worker-Web Role Comms </li></ul><ul><li>need to add more! </li></ul><ul><li>Azure Table Storage </li></ul><ul><li>Membership </li></ul><ul><li>Stacks </li></ul>Azure Blob Storage - Image Storage
  5. 5. stacka’s twin – www.cloud dot net.com
  6. 6. www.runsaturday.com
  7. 7. runsaturday’s structure G <ul><li>RunSaturday </li></ul><ul><li>Core C# application code </li></ul>SQL Server 2005 - All data storage <ul><li>Facebook Connect </li></ul><ul><li>Membership </li></ul>Nike+, Nokia, … - Data Collection <ul><li>Web Service UI </li></ul><ul><li>Uploads from script and SportTracks </li></ul><ul><li>Main UI </li></ul><ul><li>DotNetNuke 5.0 with custom modules </li></ul><ul><li>IFrame UI </li></ul><ul><li>Blog and other presentation </li></ul><ul><li>Facebook UI </li></ul><ul><li>FBML Presentation </li></ul>Amazon EC2 - GPS Data Storage GoGrid Virtual Server(s) Google maps - Map provision
  8. 8. Azure <ul><li>In ASP.Net terms: </li></ul><ul><li>Medium trust </li></ul><ul><li>No persistent local file storage – but you can use a temp file area. </li></ul><ul><li>You cannot install anything into the GAC – assume all you have is 3.5 SP1 (4.0 “soon”) </li></ul><ul><li>You can have “worker roles” – background tasks! </li></ul><ul><li>In data terms: </li></ul><ul><li>For “structured” storage, you have “tables” – very good at storing huge amounts of data and at retrieving data using primary key – do not think “SQL” though. </li></ul><ul><li>For files you have Blob. </li></ul><ul><li>For messaging you have Queues. </li></ul><ul><li>- There are other services available… </li></ul>
  9. 9. All about scale
  10. 10. 2. Azure - Simple things are not so simple <ul><li>For example counters… </li></ul><ul><ul><li>Potentially thousands of concurrent clients writing to a single resource is a blocking point – so Azure design discourages it. </li></ul></ul><ul><ul><li>There will be patterns evolve – e.g. sharding counters can help. </li></ul></ul><ul><li>Example – keeping track of number of tagged entries in Stacka. </li></ul>
  11. 11. 3. Azure - goodbye to SQL Group By <ul><li>Also goodbye to Min, Max, Count, Average… </li></ul><ul><li>These methods just don’t scale… </li></ul><ul><li>If you want stats you have to work them out and store them </li></ul><ul><li>Google’s Map Reduce as an example – Google Analytics. </li></ul><ul><li>What’s inefficient? </li></ul><ul><li>Example – Stacka– how many stacks has a user uploaded? </li></ul>
  12. 12. 4. Azure - goodbye to transactions. Hello to queues and optimistic updates. <ul><li>For speed of service, Azure tables only support optimistic locking. </li></ul><ul><li>Your design needs to work out how to deal with conflicts. </li></ul><ul><li>Again, new patterns are needed and will evolve. </li></ul><ul><li>Example - stacka – deleting a stack… the cross-table steps involved… possible patterns to solve this… </li></ul>
  13. 13. 5. Azure – get used to waiting <ul><li>The deployment environment, the development environment and the tools and libraries provided by Microsoft are all very advanced and do provide a really rich environment. </li></ul><ul><li>But… it can take a long, long, long time to deploy and put live a deployment on Azure! </li></ul><ul><li>When using the portal, take it *slow* </li></ul>
  14. 14. But this is not complaining… <ul><li>The CTP release is feature rich and it works! </li></ul><ul><li>There are plenty of things you can do: </li></ul><ul><ul><li>Develop on your desktop using all your normal tools. </li></ul></ul><ul><ul><li>Debug use either local SQL Server cloud emulation – or you can develop against real cloud storage </li></ul></ul><ul><ul><li>Use your existing .Net code </li></ul></ul><ul><ul><li>Use AJAX (Atlas) including the Ajax toolkit </li></ul></ul><ul><ul><li>Use both Web and Worker Roles </li></ul></ul><ul><ul><li>There’s first stage analytics – and logging - available </li></ul></ul><ul><ul><li>Use simultaneous production and staging servers – to ease testing of new releases. </li></ul></ul><ul><ul><li>Develop truly scalable solutions </li></ul></ul><ul><ul><li>Develop new ideas, new technologies, new businesses. </li></ul></ul>
  15. 15. Still all about scale
  16. 16. Using GoGrid, Amazon,… <ul><li>Very different to Azure and GAE. </li></ul><ul><li>GoGrid and EC2 give you access to virtual servers – whole Win2k3 or Win2k8 (or Linux) servers running as virtual PCs on real servers. </li></ul><ul><li>Very quick and easy to set up or tear down additional servers. </li></ul><ul><li>I chose GoGrid based on cost. </li></ul>
  17. 17. 6. GoGrid Server – what you get
  18. 18. 7. Using AWS S3 <ul><li>Really scalable storage solution for files (blobs) – same as Azure Blobs – don’t know how pricing will compare? </li></ul><ul><li>ThreeSharp on CodePlex – simple, straight-forward – superb. </li></ul>public string AddActivity(Activity_t activity) { Guid g = Guid.NewGuid(); string keyName = g.ToString(&quot;N&quot;); // N is guid as hex with no punctuation :) string activityAsText = TCXParser.TCXParser.ActivityToString(activity);   ThreeSharpWrapper wrapper = new ThreeSharpWrapper(AwsAccessKey, AwsSecretKey); wrapper.AddStringObjectWithCompression(AwsActivityBucketName, keyName, activityAsText);   return keyName; } public Activity_t GetActivity(string activityKeyName) { ThreeSharpWrapper wrapper = new ThreeSharpWrapper(AwsAccessKey, AwsSecretKey); string activityAsText = wrapper.GetStringObjectWithDecompression(AwsActivityBucketName, activityKeyName); return TCXParser.TCXParser.StringToActivity(activityAsText); }
  19. 19. The clock is ticking…
  20. 20. 8. Facebook Connect <ul><li>Easy way to get users to register on your site. </li></ul><ul><li>Can be integrated with ASP.Net Membership (and with DotNetNuke membership) </li></ul><ul><li>A way to get access to more publicity – to more potential users. </li></ul><ul><li>But… </li></ul><ul><ul><li>facebook is not a “’stable” platform… </li></ul></ul>
  21. 21. 9. DotNetNuke – Look Again <ul><li>Powerful .Net CMS. </li></ul><ul><li>5.0 is 10x faster than I remember 4.0 </li></ul><ul><li>Commercial module availability good. </li></ul><ul><li>Developing modules internally using Linq, 3.5, AJAX, etc are all easy – as easy as developing controls for normal ASP.Net </li></ul><ul><li>100% access to source code (almost) </li></ul><ul><li>Instant tools in your application – like Log Viewer, User Manager, SQL editor, etc </li></ul><ul><li>But… </li></ul><ul><ul><li>Not suitable for Azure… </li></ul></ul>
  22. 22. 10. Microsoft Chart Control <ul><li>Based On Dundas. </li></ul><ul><li>Included in 4.0 </li></ul><ul><li>Simple add-on to 3.5 SP1. </li></ul><ul><li>Powerful. Fast. Good. </li></ul>
  23. 23. www.clouddotnet.com www.stacka.com www.runsaturday.com slodge.blogspot.com …

×