10 things I’ve learnt In the clouds

Uploaded on

Presentation on some of my recent experiences with Microsoft Azure, Amazon S3, GoGrid and other cloud technologies, especially while developing: …

Presentation on some of my recent experiences with Microsoft Azure, Amazon S3, GoGrid and other cloud technologies, especially while developing:
- http://www.runsaturday.com
- http://www.stacka.com
- http://www.clouddotnet.com

I'm presenting this tonight at the London .Net User's group - but thought it would be useful to share more widely!

If you need more info, contact me@slodge.com - please mark your email with No Spam somehow... hopefully it will get through to me.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. 10 Things I’ve learnt in the cloud Stuart Lodge [email_address] http://slodge.blogspot.com
  • 2. 1. There’s more than one cloud
    • “ Software As A Service”
    • Scalable Storage – blobs, queues, tables – S3, Azure
    • Data services
    • Membership services – Live, Google, OpenId, Facebook Connect, …
    • Virtual application hosts (GAE and Azure)
    • Virtual servers and virtual shared hosting (EC2, GoGrid, Mosso, …)
  • 3. www.stacka.com
  • 4. stacka’s structure
    • Stacka
    • Azure WebRole (like a ASP.NET web app)
    • Azure WorkerRole (like a .NET command line app)
    • Azure Queue Storage
    • Worker-Web Role Comms
    • need to add more!
    • Azure Table Storage
    • Membership
    • Stacks
    Azure Blob Storage - Image Storage
  • 5. stacka’s twin – www.cloud dot net.com
  • 6. www.runsaturday.com
  • 7. runsaturday’s structure G
    • RunSaturday
    • Core C# application code
    SQL Server 2005 - All data storage
    • Facebook Connect
    • Membership
    Nike+, Nokia, … - Data Collection
    • Web Service UI
    • Uploads from script and SportTracks
    • Main UI
    • DotNetNuke 5.0 with custom modules
    • IFrame UI
    • Blog and other presentation
    • Facebook UI
    • FBML Presentation
    Amazon EC2 - GPS Data Storage GoGrid Virtual Server(s) Google maps - Map provision
  • 8. Azure
    • In ASP.Net terms:
    • Medium trust
    • No persistent local file storage – but you can use a temp file area.
    • You cannot install anything into the GAC – assume all you have is 3.5 SP1 (4.0 “soon”)
    • You can have “worker roles” – background tasks!
    • In data terms:
    • For “structured” storage, you have “tables” – very good at storing huge amounts of data and at retrieving data using primary key – do not think “SQL” though.
    • For files you have Blob.
    • For messaging you have Queues.
    • - There are other services available…
  • 9. All about scale
  • 10. 2. Azure - Simple things are not so simple
    • For example counters…
      • Potentially thousands of concurrent clients writing to a single resource is a blocking point – so Azure design discourages it.
      • There will be patterns evolve – e.g. sharding counters can help.
    • Example – keeping track of number of tagged entries in Stacka.
  • 11. 3. Azure - goodbye to SQL Group By
    • Also goodbye to Min, Max, Count, Average…
    • These methods just don’t scale…
    • If you want stats you have to work them out and store them
    • Google’s Map Reduce as an example – Google Analytics.
    • What’s inefficient?
    • Example – Stacka– how many stacks has a user uploaded?
  • 12. 4. Azure - goodbye to transactions. Hello to queues and optimistic updates.
    • For speed of service, Azure tables only support optimistic locking.
    • Your design needs to work out how to deal with conflicts.
    • Again, new patterns are needed and will evolve.
    • Example - stacka – deleting a stack… the cross-table steps involved… possible patterns to solve this…
  • 13. 5. Azure – get used to waiting
    • The deployment environment, the development environment and the tools and libraries provided by Microsoft are all very advanced and do provide a really rich environment.
    • But… it can take a long, long, long time to deploy and put live a deployment on Azure!
    • When using the portal, take it *slow*
  • 14. But this is not complaining…
    • The CTP release is feature rich and it works!
    • There are plenty of things you can do:
      • Develop on your desktop using all your normal tools.
      • Debug use either local SQL Server cloud emulation – or you can develop against real cloud storage
      • Use your existing .Net code
      • Use AJAX (Atlas) including the Ajax toolkit
      • Use both Web and Worker Roles
      • There’s first stage analytics – and logging - available
      • Use simultaneous production and staging servers – to ease testing of new releases.
      • Develop truly scalable solutions
      • Develop new ideas, new technologies, new businesses.
  • 15. Still all about scale
  • 16. Using GoGrid, Amazon,…
    • Very different to Azure and GAE.
    • GoGrid and EC2 give you access to virtual servers – whole Win2k3 or Win2k8 (or Linux) servers running as virtual PCs on real servers.
    • Very quick and easy to set up or tear down additional servers.
    • I chose GoGrid based on cost.
  • 17. 6. GoGrid Server – what you get
  • 18. 7. Using AWS S3
    • Really scalable storage solution for files (blobs) – same as Azure Blobs – don’t know how pricing will compare?
    • ThreeSharp on CodePlex – simple, straight-forward – superb.
    public string AddActivity(Activity_t activity) { Guid g = Guid.NewGuid(); string keyName = g.ToString("N"); // N is guid as hex with no punctuation :) string activityAsText = TCXParser.TCXParser.ActivityToString(activity);   ThreeSharpWrapper wrapper = new ThreeSharpWrapper(AwsAccessKey, AwsSecretKey); wrapper.AddStringObjectWithCompression(AwsActivityBucketName, keyName, activityAsText);   return keyName; } public Activity_t GetActivity(string activityKeyName) { ThreeSharpWrapper wrapper = new ThreeSharpWrapper(AwsAccessKey, AwsSecretKey); string activityAsText = wrapper.GetStringObjectWithDecompression(AwsActivityBucketName, activityKeyName); return TCXParser.TCXParser.StringToActivity(activityAsText); }
  • 19. The clock is ticking…
  • 20. 8. Facebook Connect
    • Easy way to get users to register on your site.
    • Can be integrated with ASP.Net Membership (and with DotNetNuke membership)
    • A way to get access to more publicity – to more potential users.
    • But…
      • facebook is not a “’stable” platform…
  • 21. 9. DotNetNuke – Look Again
    • Powerful .Net CMS.
    • 5.0 is 10x faster than I remember 4.0
    • Commercial module availability good.
    • Developing modules internally using Linq, 3.5, AJAX, etc are all easy – as easy as developing controls for normal ASP.Net
    • 100% access to source code (almost)
    • Instant tools in your application – like Log Viewer, User Manager, SQL editor, etc
    • But…
      • Not suitable for Azure…
  • 22. 10. Microsoft Chart Control
    • Based On Dundas.
    • Included in 4.0
    • Simple add-on to 3.5 SP1.
    • Powerful. Fast. Good.
  • 23. www.clouddotnet.com www.stacka.com www.runsaturday.com slodge.blogspot.com …