Cloudtime - Adventures in Microsoft Azure

  • 2,422 views
Uploaded on

A draft white paper on Azure cloud development for http://www.stacka.com

A draft white paper on Azure cloud development for http://www.stacka.com

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • please send me this ppt on kineticmayank@yahoo.com
    Are you sure you want to
    Your message goes here
    Be the first to like this
No Downloads

Views

Total Views
2,422
On Slideshare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
0
Comments
1
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Cloudtime – Adventures with Azure A few months ago, I saw a few people comparing their book collections online. What they did was to take all of their books on one topic, to put them in one big pile, and then take a picture of them. The result was something like the “stack” you can see on the right. What a fabulous idea! If only there was a website where people could do that … a place maybe called www.stacka.com? But how would such a website work? • It would need a web page UI – including things like AJAX special effects. • It would need server side application logic to process the UI inputs and to deliver the web pages. • It would need some form of database on the server – especially to handle information like website membership and like the index of “stacks”. • It would need server side storage of the photos. And obviously, because stacka was going to be a huge success, whatever I built would need to scale – it would need to be built so that millions of users could all use it at the same time. And just as I was thinking this, Microsoft announced Azure – their new “cloud” operating system – perfect timing – lets build! Figure 1. My travel book stack
  • 2. Building the UI I’m not really a web developer… in fact, it wouldn’t be too far from the truth to say that I’ve not used ASP since 1998. However, after a quick tour through some tutorials and videos and after a few stumbles through the wizards, I jumped on into Visual Studio 2008 and started plumbing a first UI together, including: • Some new ASPX web forms to serve the pages • An ASP master page for me to really show off my design skills (if only I had any!) • Some new ASCX custom user controls to help structure my code • Lots of existing server controls – especially things like Login controls - so that I didn’t have to write these from scratch. • Some AJAX Toolkit dynamic controls – especially so I could make my pages look a bit less clunky and a bit more “2.0” • Some data binding – I do love writing code, but drag and drop is perfect for repetitive tasks. And that was it – the UI all done – nothing special for Azure except that it was named “web role” rather than “website” – other than that, it is just standard components inside a standard ASP website. Figure 2. Azure web development – normal ASP.Net code
  • 3. Application Logic Stacka’s not a particularly complicated application – it just provides a few functions like Add, Browse, Rate, and Comment. I think that’s what’s called Business Logic? And this logic needs to go somewhere. The best place for it seemed to be in a class library, and so that’s what I wrote: a C# library using all the normal goodies within .Net 3.5 – generics, lambdas, linq, etc. – and pulling in as much functionality as possible from existing code – from the System libraries, from the built-in Membership pattern, and from the Azure Samples libraries supplied by Microsoft. Figure 3 Azure class library development – normal class library development
  • 4. Some form of database… I knew www.stacka.com would be a huge success – I knew there would soon be millions of users signed up – and that these millions of users would each be adding hundreds of stacks to the site. How would I store such a large list of stacks? How would I index and access that storage? Azure presented me with three basic choices: 1. I could use Azure Table Storage – a structured data store providing fast access for millions of rows with simple search facilities, but (because it would slow down the access speeds) with no in-built ability to do “advanced” data manipulation such as join, order by, sum, or group by. 2. I could use SQL Data Services – a structured data store again providing fast access for millions of rows, but with more “advanced” functionality available – but still a long way from full SQL Server functionality, and perhaps delivered at a cost of both in terms of slower access speeds and increased license costs. 3. I could use an external datasource – maybe an SQL database or a set of WCF web services – but this would have to be hosted on some other server. For stacka, I chose option 1 - pure Azure. This selection did have some big impacts on my data design and back on my application logic – e.g. because I could not rely on a database to do “joins” then I decided to break away from SQL normalisation –to deliberately include some redundant data in the store. This may sound wasteful, but at the time it felt like a normal “Cycles-vs-Memory” decision - it was a planned choice to consume extra storage in order to consume last cycles – a choice that a “faster” UI was more important than a more compact database. The code for using the database store is all class based. Any object (like a Stack, a Comment or a Rating) which stacka persists: • is defined in code as a standard C# class • is inherited from TableStorageEntity • exposes two special string public properties to form the primary key – PartitionKey and RowKey. • exposes other public properties as persistent properties - including types such as String, DateTime, Int32, Double, etc • can be newed, loaded and saved using HTTP or HTTPS calls to the Azure Table storage – in stacka, these are wrapped using Microsoft’s “Microsoft.Samples.ServiceHosting.StorageClient” code – which allows the use of Linq queries – e.g. to get all the stacks for a user.
  • 5. /// <summary> /// Return stacks for a given user /// </summary> /// <param name=quot;userIdquot;></param> /// <returns></returns> public static List<StackRow> GetStacksForUser(string userId) { TableStorageDataServiceContext svc = CreateDataServiceContext(); IEnumerable<StackRow> query = from c in svc.CreateQuery<StackRow>(_tableName) where c.UserId == userId select c; TableStorageDataServiceQuery<StackRow> q = new TableStorageDataServiceQuery<StackRow> (query as DataServiceQuery<StackRow>, _tableRetry); IEnumerable<StackRow> stackRows = q.ExecuteAllWithRetries(); if (stackRows != null) { List<StackRow> list = new List<StackRow>(stackRows); return list; } return null; } Figure 4. The C# code to get the stacks for a given user Note: the selection of PartitionKey and RowKey is very important – this selection determines the indexing, storage and ordering of objects within a type across the Azure Table store. Depending on what you choose to use, this will determine the lookup speed and order for various queries. It’s a very detailed topic and I won’t go into it further in this paper – for more information, try: http://slodge.blogspot.com/search? q=partitionkey, http://blog.smarx.com/ and http://msdn.microsoft.com/en-us/library/dd179338.aspx
  • 6. Storing the stack photos Stacka lets its users upload high resolution images of their stacks. Obviously each of these images could be quite large. The data within these images also doesn’t really need indexing – it’s just binary data that needs to be returned to a web browser occasionally. To store these blobs, stacka uses the Azure Blob Storage – this store allows: • The stacka application to make simple HTTP or HTTPS calls to store, retrieve or delete blobs (files) • The stacka application to make blobs as public – so that files can be referenced by hyperlink within web pages – e.g. inside <img> or tags. As with the Table storage, stacka uses the “Microsoft.Samples.ServiceHosting.StorageClient” code to wrap access to the Blob storage: public static string CreateBloblmage(string contentType, Stream content) { BlobStorage blobStorage = BlobStorage.Create( StorageAccountInfo.GetDefaultBlobStorageAccountFromConfiguration()); BlobContainer blobContainer = blobStorage.GetBlobContainer(quot;photogalleryquot;); BlobProperties props = new BlobProperties( string.Format(quot;{0:10}_{1}quot;, DateTime.MaxValue.Ticks - DateTime.Now.Ticks, Guid.NewGuid())) { ContentType = contentType }; blobContainer.CreateBlob( props, new BlobContents(content), true); return props.Name; } Figure 5. Example code to access the blob storage If a typical photo was 1MB, and a typical user uploaded 100 photos, then when stacka lands it’s one millionth customer, stacka will have 100000000MB of data to worry about. As a one person startup, I’m glad that backing up and serving that photo library will be someone else’s problem – that’s the cloud’s job!
  • 7. Improving UI performance – adding a worker role Storing images as 3MB files may be easy in the cloud, but it doesn’t really make for a snappy user interface. To help with this, stacka generates thumbnail images to serve in place of the main image. In order to improve the perceived UI performance, these thumbnails are not generated directly by the upload code – but are instead generated by a background task – called a “worker role”. You can think of this worker role as a bit like a command line application or windows service – it’s implemented as a normal C# class running with a “forever” message loop. This message loop gets a message from the main web front end whenever a user uploads a new image – and then performs the necessary thumbnail generation – with the thumbnails themselves stored. The messaging mechanism used is an Azure Queue – a simple, scalable service allowing multiple web user interfaces (or worker roles or external applications) to send and receive strings (stacka uses objects serialized as xml for these strings). Confused? Maybe (hopefully) the diagrams below might help explain the interactions. Figure 6 Web UI interaction with Azure
  • 8. Figure 7 Worker role interaction with Azure Figure 8 Web browser interaction with Web UI and with Azure
  • 9. Development and Deployment The Azure Community Technology Preview is an add-on to Visual Studio 2008. This add on includes: • A development “storage controller” built on SQL Server Express and imitating the Table, Blob and Queue server. • A development “fabric controller” built around IIS7 and providing hosting of both web and worker roles. What this meant was that all stacka development could be done on the desktop in my normal VS2008 tools, although admittedly only Vista will work – no XP and no Windows 7. Once I was happy with the initial implementation, then I could then start working on the real Azure cloud – I could do this in two ways: 1. I could switch the storage from the local storage controller to the real Azure storage – this allowed me to run my code in the debugger while using the real Azure Table, Blob and Queue servers. 2. I could deploy both the UI and the storage to the real Azure servers – once there debugging is at a logging level only – but this is like being out there for real. The tools for doing this second step are already quite sophisticated – they include a “Publish…” assistant inside Visual Studio and the Azure service website which allows you to host both a staging and a live deployment. However, the process of deploying is not very quick at present – expect it to take at least an hour to get your first web app up and running – almost all of which is spent waiting for steps to complete and processes to start…
  • 10. Beyond Stacka Version 1 Stacka Version 1 is up and live on http://www.stacka.com. It’s not yet reached 1 million users – but I’m sure you’ll all be signing up and adding your stacks soon! Once Stacka was up and running, I decided to try to play with some of the features a little – including adding more review and comment exposure. The outcome was http://www.clouddotnet.com – the leading directory of Azure powered websites – take a look around to see what other people are doing on Azure. And then it snowed… so I build http://www.snowdotnet.com in addition to the other two apps. One interesting thing here is that snowdotnet itself is not hosted on Azure. Because I’d run out of tokens from Microsoft I hosted just the Storage on Azure – the web UI itself is not Azure hosted – it’s on a conventional Windows 2008 server running IIS7. So mashups where the UI is conventional but the scalable storage in on Azure are not only possible – they already exist! And there’s plenty more to play with yet… watch this space!
  • 11. About Me Stuart Lodge – studied maths at Cambridge a long time ago – developed lots of cool code for lots of years – occasionally found training for my serial marathon and ironman habits Currently not really employed, but to be found having lots of fun working on: – Sporty developments like: http://www.runsaturday.com – sign up today o – Azure developments like : http://www.stacka.com o http://www.snowdotnet.com o http://www.clouddotnet.com o – My techie blog: http://slodge.blogspot.com o – On twitter: @slodge o – On linked in: http://www.linkedin.com/in/stuartlodge o – Generally online and on the cloud! If you’re interested in the cloud – please come and say hello!