Hands On Windows Azure  Building A Twitter Clone
Upcoming SlideShare
Loading in...5
×
 

Hands On Windows Azure Building A Twitter Clone

on

  • 2,724 views

Slide deck from my talk at the SDN Conference in Amsterdam in Fall 2009

Slide deck from my talk at the SDN Conference in Amsterdam in Fall 2009

Statistics

Views

Total Views
2,724
Views on SlideShare
2,504
Embed Views
220

Actions

Likes
1
Downloads
21
Comments
0

5 Embeds 220

http://www.kalyani.com 150
http://kalyani.com 66
http://www.slideshare.net 2
http://b.hatena.ne.jp 1
http://feeds2.feedburner.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Hands On Windows Azure  Building A Twitter Clone Hands On Windows Azure Building A Twitter Clone Presentation Transcript

    • Nik Kalyani
      DotNetNuke Corporation
      Windows Azure: Building a Twitter Clone
      Session Code: NE.09
    • About
      Nik Kalyani
      • Co-founder, DotNetNuke Corporation
      • Technology Entrepreneur, 15 years
      • Microsoft MVP (2005-2009)
      Online
      • Email: nik@kalyani.com
      • Twitter: @techbubble
      • Blog: http://www.techbubble.net
    • Session Objective
      Provide insights on how to architect a large-scale, asynchronous application with the same capabilities as Twitter on Azure while leveraging its non-RDMS capabilities
    • Agenda
      Azure Architecture
      Storage Concepts
      Micro-blogging Application
      Requirements
      Architecture
      Processing
    • Azure Architecture
      Your Applications

      ServiceBus
      Workflow
      Database
      Analytics
      Identity
      Contacts
      AccessControl

      Reporting

      Devices

      Compute
      Storage
      Manage

    • Azure Application Architecture
      n
      m
      Worker Role
      Web Role
      LB
      Cloud Storage (blob, table, queue)
    • Azure Durable Storage
      Durable, scalable, available store
      Simple interface
      REST
      ADO.NET Data Services
      Simple abstractions
      Blobs
      Tables
      Queues

    • Azure Data Storage Concepts
      Container
      Blobs
      http://<account>.blob.core.windows.net/<container>
      Table
      Entities
      Account
      http://<account>.table.core.windows.net/<table>
      Queue
      Messages
      http://<account>.queue.core.windows.net/<queue>
    • Block
      Blob
      Container
      Account
      Blob Storage Concepts
      IMG001.JPG
      pictures
      IMG002.JPG
      erica
      Block 1
      movies
      MOV1.AVI
      Block 2
      Block 3
    • Blobs: Features and Functions
      Store large objects
      Associate metadata with blob
      Metadata is <name, value> pairs
      Set/Get with or separate from blob data bits
      Standard REST Interface
      Limitations
      Block Id - 64 Bytes Per Block
      PutBlob - 64 MB Total
      Blob MetaData - 8 KB Per Blob
      PutBlock / PutBlockList - 50 GB Each
      Blocks - 4 MB Each
    • Table Storage Concepts
      Entity
      Table
      Account
      Name =…
      Email = …
      users
      Name =…
      Email = …
      erica
      Photo ID =…
      Date =…
      photo index
      Photo ID =…
      Date =…
    • Tables: Features and Functions
      Massively scalable, highly available and durable structured storage
      Not RDMS -- no joins, no maintenance of foreign keys
      LINQ and REST interfaces via ADO.Net Data Services
      Partition key is exposed to the application
      Limitations
      Up to 255 Entity Properties (including PartitionKey, RowKey, and Timestamp)
      1 MB Max for Combined Size of All Data in an Entity's Properties
    • Queue Storage Concepts
      Message
      Queue
      Account
      128x128, http://…
      thumbnail jobs
      256x256, http://…
      erica
      http://…
      photo processing jobs
      http://…
    • Queues: Features and Functions
      An account can create many queues..
      Queue Name is scoped by the account
      A Queue contains messages..
      No limit on number of messages stored in a queue
      A message is stored for at most a week in a queue
      http://<Account>.queue.core.windows.net/<QueueName>
      Messages..
      Message size <= 8 KB
      To store larger data, store data in blob/entity storage, and the blob/entity name in the message
    • Micro-blogging ApplicationRequirements
      Relationships
      Any user can “follow” any other user
      A user can “unfollow” another user he/she is “following”
    • Micro-blogging ApplicationRequirements
      Micro-blogging
      Any user can post a message of up to 140 characters in length
      All message posts are public
    • Micro-blogging ApplicationRequirements
      Timeline
      A public timeline of all posted messages is available to anybody for up to 24 hours from the time the message was first posted
      A user timeline of all messages posted by a user is available to anybody forever
      A “following” timeline of all messages posted by all users that a user is following in the prior 24 hours is available to each user
    • Micro-blogging ApplicationArchitectural Considerations
      Micro-blogging
      The number of users will determine volume of messages
      However, common situation is that a small number of users will post majority of the messages
    • Micro-blogging ApplicationArchitectural Considerations
      Timelines
      The public timeline is transient and will have a high frequency of reads and writes
      A user timeline is permanent and will have a small number of writes and a large number of reads
      A “following” timeline is transient and will have a large number of irregularly timed writes and a small number of reads
    • Micro-blogging ApplicationProfile, Messages and Relationships Storage
      Table = User
      PartitionKey = Username
      RowKey=string.Format("{0:D19}",
      DateTime.MaxValue.Ticks –
      DateTime.UtcNow.Ticks);
      Use Table storage sharded by user
      Entities with the same PartitionKey are stored on the same storage node
      Obtaining recent messages, followers or following list require only a single query for each
      Adding or Deleting Follower or Following requires two writes (i.e. one for each user)
    • Micro-blogging ApplicationFollow
      User B Partition
      User A follows User B
      Following
      [User D]
      Message Queue
      Maintenance Queue
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User C]
    • Micro-blogging ApplicationFollow
      User B Partition
      Follower
      [User A]
      Following
      [User D]
      Message Queue
      Maintenance Queue
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User C]
      Following
      [User B]
    • Micro-blogging ApplicationFollow
      User B Partition
      User B follows User A
      Following
      [User D]
      Follower
      [User A]
      Message Queue
      Maintenance Queue
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User C]
      Following
      [User B]
    • Micro-blogging ApplicationFollow
      User B Partition
      Follower
      [User A]
      Following
      [User A]
      Following
      [User D]
      Message Queue
      Maintenance Queue
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User B]
      Follower
      [User C]
      Following
      [User B]
    • Micro-blogging ApplicationMessage Posting
      User B Partition
      User A posts a message
      Follower
      [User A]
      Following
      [User A]
      Following
      [User D]
      Message Queue
      Maintenance Queue
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User B]
      Follower
      [User C]
      Following
      [User B]
    • Micro-blogging ApplicationMessage Posting
      User B Partition
      Message added to Message Queue
      Follower
      [User A]
      Following
      [User A]
      Following
      [User D]
      Message Queue
      User A
      Message
      Maintenance Queue
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User B]
      Follower
      [User C]
      Following
      [User B]
    • Micro-blogging ApplicationMessage Posting
      User B Partition
      After Message Worker Role processes…
      Follower
      [User A]
      Following
      [User A]
      Following
      [User D]
      Message Queue
      Msg (A)
      Maintenance Queue
      User B
      User C
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User B]
      Follower
      [User C]
      Following
      [User B]
      Msg (A)
      Msg (A)
    • Micro-blogging ApplicationMessage Posting
      User B Partition
      After Maintenance Worker Role processes…
      Follower
      [User A]
      Following
      [User A]
      Following
      [User D]
      Message Queue
      Msg (A)
      Maintenance Queue
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User B]
      Follower
      [User C]
      Following
      [User B]
      Msg (A)
      Msg (A)
    • Micro-blogging ApplicationMessage Maintenance
      User B Partition
      User A posts a message
      Follower
      [User A]
      Following
      [User A]
      Following
      [User D]
      Message Queue
      Msg (A)
      Msg (D)
      Msg (D)
      25 hours
      Maintenance Queue
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User B]
      Follower
      [User C]
      Following
      [User B]
      Msg (A)
      Msg (X)
      Msg (A)
    • Micro-blogging ApplicationMessage Maintenance
      User B Partition
      Message added to Message Queue
      Follower
      [User A]
      Following
      [User A]
      Following
      [User D]
      Message Queue
      User A
      Message
      Msg (A)
      Msg (D)
      Msg (D)
      25 hours
      Maintenance Queue
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User B]
      Follower
      [User C]
      Following
      [User B]
      Msg (A)
      Msg (X)
      Msg (A)
    • Micro-blogging ApplicationMessage Maintenance
      User B Partition
      After Message Worker Role processes…
      Follower
      [User A]
      Following
      [User A]
      Following
      [User D]
      Message Queue
      Msg (A)
      Msg (D)
      Msg (D)
      25 hours
      Msg (A)
      Maintenance Queue
      User B
      User C
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User B]
      Follower
      [User C]
      Following
      [User B]
      Msg (A)
      Msg (X)
      Msg (A)
      Msg (A)
      Msg (A)
    • Micro-blogging ApplicationMessage Maintenance
      User B Partition
      After Maintenance Worker Role processes…
      Follower
      [User A]
      Following
      [User A]
      Following
      [User D]
      Message Queue
      Msg (A)
      Msg (D)
      Msg (A)
      Maintenance Queue
      User C Partition
      User A Partition
      Follower
      [User Y]
      Follower
      [User Z]
      Following
      [User A]
      Following
      [User X]
      Follower
      [User B]
      Follower
      [User C]
      Following
      [User B]
      Msg (A)
      Msg (X)
      Msg (A)
      Msg (A)
      Msg (A)
    • Micro-blogging ApplicationTakeaways
      Cloud app storage significantly different from RDMS…requires shift in thinking
      Queues make apps efficient … processing is done asynchronously
      Duplication of data is often required and this is OK
      Sharding is an important consideration for the scalability of your cloud application
      Absence of join, sort make application design challenging but with planning this is manageable
    • Evaluation form
      Vul je evaluatieformulier in en maak kans op een van de prachtige prijzen!!
      Fill out your evaluation form and win one of the great prizes!!
      Session Code: NE.09