Your SlideShare is downloading. ×
0
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Hands On Windows Azure  Building A Twitter Clone
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Hands On Windows Azure Building A Twitter Clone

2,114

Published on

Slide deck from my talk at the SDN Conference in Amsterdam in Fall 2009

Slide deck from my talk at the SDN Conference in Amsterdam in Fall 2009

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,114
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Nik Kalyani<br />DotNetNuke Corporation<br />Windows Azure: Building a Twitter Clone<br />Session Code: NE.09<br />
  • 2. About<br />Nik Kalyani<br /><ul><li>Co-founder, DotNetNuke Corporation
  • 3. Technology Entrepreneur, 15 years
  • 4. Microsoft MVP (2005-2009)</li></ul>Online <br /><ul><li>Email: nik@kalyani.com
  • 5. Twitter: @techbubble
  • 6. Blog: http://www.techbubble.net</li></li></ul><li>Session Objective<br />Provide insights on how to architect a large-scale, asynchronous application with the same capabilities as Twitter on Azure while leveraging its non-RDMS capabilities<br />
  • 7. Agenda<br />Azure Architecture<br />Storage Concepts<br />Micro-blogging Application <br />Requirements<br />Architecture<br />Processing<br />
  • 8. Azure Architecture<br />Your Applications<br />…<br />ServiceBus<br />Workflow<br />Database<br />Analytics<br />Identity<br />Contacts<br />AccessControl<br />…<br />Reporting<br />…<br />Devices<br />…<br />Compute<br />Storage<br />Manage<br />…<br />
  • 9. Azure Application Architecture<br />n<br />m<br />Worker Role<br />Web Role<br />LB<br />Cloud Storage (blob, table, queue)<br />
  • 10. Azure Durable Storage<br />Durable, scalable, available store<br />Simple interface<br />REST<br />ADO.NET Data Services<br />Simple abstractions<br />Blobs<br />Tables<br />Queues<br />…<br />
  • 11. Azure Data Storage Concepts<br /> Container<br />Blobs<br />http://&lt;account&gt;.blob.core.windows.net/&lt;container&gt;<br />Table<br />Entities<br />Account<br />http://&lt;account&gt;.table.core.windows.net/&lt;table&gt;<br />Queue<br />Messages<br />http://&lt;account&gt;.queue.core.windows.net/&lt;queue&gt;<br />
  • 12. Block<br />Blob<br />Container<br />Account<br />Blob Storage Concepts<br />IMG001.JPG<br />pictures<br />IMG002.JPG<br />erica<br />Block 1<br />movies<br />MOV1.AVI<br />Block 2<br />Block 3<br />
  • 13. Blobs: Features and Functions<br />Store large objects <br />Associate metadata with blob<br />Metadata is &lt;name, value&gt; pairs<br />Set/Get with or separate from blob data bits<br />Standard REST Interface<br />Limitations<br />Block Id - 64 Bytes Per Block<br />PutBlob - 64 MB Total<br />Blob MetaData - 8 KB Per Blob<br />PutBlock / PutBlockList - 50 GB Each<br />Blocks - 4 MB Each<br />
  • 14. Table Storage Concepts<br />Entity<br />Table<br />Account<br />Name =…<br />Email = …<br />users<br />Name =…<br />Email = …<br />erica<br />Photo ID =…<br />Date =…<br />photo index<br />Photo ID =…<br />Date =…<br />
  • 15. Tables: Features and Functions<br />Massively scalable, highly available and durable structured storage<br />Not RDMS -- no joins, no maintenance of foreign keys<br />LINQ and REST interfaces via ADO.Net Data Services<br />Partition key is exposed to the application<br />Limitations<br />Up to 255 Entity Properties (including PartitionKey, RowKey, and Timestamp)<br />1 MB Max for Combined Size of All Data in an Entity&apos;s Properties<br />
  • 16. Queue Storage Concepts<br />Message<br />Queue<br />Account<br />128x128, http://…<br />thumbnail jobs<br />256x256, http://…<br />erica<br />http://…<br />photo processing jobs<br />http://…<br />
  • 17. Queues: Features and Functions<br />An account can create many queues..<br />Queue Name is scoped by the account<br />A Queue contains messages..<br />No limit on number of messages stored in a queue<br />A message is stored for at most a week in a queue<br />http://&lt;Account&gt;.queue.core.windows.net/&lt;QueueName&gt;<br />Messages..<br />Message size &lt;= 8 KB<br />To store larger data, store data in blob/entity storage, and the blob/entity name in the message<br />
  • 18. Micro-blogging ApplicationRequirements<br />Relationships<br />Any user can “follow” any other user<br />A user can “unfollow” another user he/she is “following”<br />
  • 19. Micro-blogging ApplicationRequirements<br />Micro-blogging<br />Any user can post a message of up to 140 characters in length<br />All message posts are public<br />
  • 20. Micro-blogging ApplicationRequirements<br />Timeline<br />A public timeline of all posted messages is available to anybody for up to 24 hours from the time the message was first posted<br />A user timeline of all messages posted by a user is available to anybody forever<br />A “following” timeline of all messages posted by all users that a user is following in the prior 24 hours is available to each user<br />
  • 21. Micro-blogging ApplicationArchitectural Considerations<br />Micro-blogging<br />The number of users will determine volume of messages<br />However, common situation is that a small number of users will post majority of the messages<br />
  • 22. Micro-blogging ApplicationArchitectural Considerations<br />Timelines<br />The public timeline is transient and will have a high frequency of reads and writes<br />A user timeline is permanent and will have a small number of writes and a large number of reads<br />A “following” timeline is transient and will have a large number of irregularly timed writes and a small number of reads<br />
  • 23. Micro-blogging ApplicationProfile, Messages and Relationships Storage<br />Table = User<br />PartitionKey = Username<br />RowKey=string.Format(&quot;{0:D19}&quot;, <br />DateTime.MaxValue.Ticks – <br />DateTime.UtcNow.Ticks);<br />Use Table storage sharded by user<br />Entities with the same PartitionKey are stored on the same storage node<br />Obtaining recent messages, followers or following list require only a single query for each<br />Adding or Deleting Follower or Following requires two writes (i.e. one for each user)<br />
  • 24. Micro-blogging ApplicationFollow<br />User B Partition<br />User A follows User B<br />Following<br />[User D]<br />Message Queue<br />Maintenance Queue<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User C]<br />
  • 25. Micro-blogging ApplicationFollow<br />User B Partition<br />Follower<br />[User A]<br />Following<br />[User D]<br />Message Queue<br />Maintenance Queue<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User C]<br />Following<br />[User B]<br />
  • 26. Micro-blogging ApplicationFollow<br />User B Partition<br />User B follows User A<br />Following<br />[User D]<br />Follower<br />[User A]<br />Message Queue<br />Maintenance Queue<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User C]<br />Following<br />[User B]<br />
  • 27. Micro-blogging ApplicationFollow<br />User B Partition<br />Follower<br />[User A]<br />Following<br />[User A]<br />Following<br />[User D]<br />Message Queue<br />Maintenance Queue<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User B]<br />Follower<br />[User C]<br />Following<br />[User B]<br />
  • 28. Micro-blogging ApplicationMessage Posting<br />User B Partition<br />User A posts a message<br />Follower<br />[User A]<br />Following<br />[User A]<br />Following<br />[User D]<br />Message Queue<br />Maintenance Queue<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User B]<br />Follower<br />[User C]<br />Following<br />[User B]<br />
  • 29. Micro-blogging ApplicationMessage Posting<br />User B Partition<br />Message added to Message Queue<br />Follower<br />[User A]<br />Following<br />[User A]<br />Following<br />[User D]<br />Message Queue<br />User A<br />Message<br />Maintenance Queue<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User B]<br />Follower<br />[User C]<br />Following<br />[User B]<br />
  • 30. Micro-blogging ApplicationMessage Posting<br />User B Partition<br />After Message Worker Role processes…<br />Follower<br />[User A]<br />Following<br />[User A]<br />Following<br />[User D]<br />Message Queue<br />Msg (A)<br />Maintenance Queue<br />User B<br />User C<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User B]<br />Follower<br />[User C]<br />Following<br />[User B]<br />Msg (A)<br />Msg (A)<br />
  • 31. Micro-blogging ApplicationMessage Posting<br />User B Partition<br />After Maintenance Worker Role processes…<br />Follower<br />[User A]<br />Following<br />[User A]<br />Following<br />[User D]<br />Message Queue<br />Msg (A)<br />Maintenance Queue<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User B]<br />Follower<br />[User C]<br />Following<br />[User B]<br />Msg (A)<br />Msg (A)<br />
  • 32. Micro-blogging ApplicationMessage Maintenance<br />User B Partition<br />User A posts a message<br />Follower<br />[User A]<br />Following<br />[User A]<br />Following<br />[User D]<br />Message Queue<br />Msg (A)<br />Msg (D)<br />Msg (D)<br />25 hours<br />Maintenance Queue<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User B]<br />Follower<br />[User C]<br />Following<br />[User B]<br />Msg (A)<br />Msg (X)<br />Msg (A)<br />
  • 33. Micro-blogging ApplicationMessage Maintenance<br />User B Partition<br />Message added to Message Queue<br />Follower<br />[User A]<br />Following<br />[User A]<br />Following<br />[User D]<br />Message Queue<br />User A<br />Message<br />Msg (A)<br />Msg (D)<br />Msg (D)<br />25 hours<br />Maintenance Queue<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User B]<br />Follower<br />[User C]<br />Following<br />[User B]<br />Msg (A)<br />Msg (X)<br />Msg (A)<br />
  • 34. Micro-blogging ApplicationMessage Maintenance<br />User B Partition<br />After Message Worker Role processes…<br />Follower<br />[User A]<br />Following<br />[User A]<br />Following<br />[User D]<br />Message Queue<br />Msg (A)<br />Msg (D)<br />Msg (D)<br />25 hours<br />Msg (A)<br />Maintenance Queue<br />User B<br />User C<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User B]<br />Follower<br />[User C]<br />Following<br />[User B]<br />Msg (A)<br />Msg (X)<br />Msg (A)<br />Msg (A)<br />Msg (A)<br />
  • 35. Micro-blogging ApplicationMessage Maintenance<br />User B Partition<br />After Maintenance Worker Role processes…<br />Follower<br />[User A]<br />Following<br />[User A]<br />Following<br />[User D]<br />Message Queue<br />Msg (A)<br />Msg (D)<br />Msg (A)<br />Maintenance Queue<br />User C Partition<br />User A Partition<br />Follower<br />[User Y]<br />Follower<br />[User Z]<br />Following<br />[User A]<br />Following<br />[User X]<br />Follower<br />[User B]<br />Follower<br />[User C]<br />Following<br />[User B]<br />Msg (A)<br />Msg (X)<br />Msg (A)<br />Msg (A)<br />Msg (A)<br />
  • 36. Micro-blogging ApplicationTakeaways<br />Cloud app storage significantly different from RDMS…requires shift in thinking<br />Queues make apps efficient … processing is done asynchronously<br />Duplication of data is often required and this is OK<br />Sharding is an important consideration for the scalability of your cloud application<br />Absence of join, sort make application design challenging but with planning this is manageable<br />
  • 37. Evaluation form<br />Vul je evaluatieformulier in en maak kans op een van de prachtige prijzen!!<br />Fill out your evaluation form and win one of the great prizes!!<br />Session Code: NE.09<br />

×